Do Your Students Know How to Analyze a Case—Really?

Explore more.

  • Case Teaching
  • Student Engagement

J ust as actors, athletes, and musicians spend thousands of hours practicing their craft, business students benefit from practicing their critical-thinking and decision-making skills. Students, however, often have limited exposure to real-world problem-solving scenarios; they need more opportunities to practice tackling tough business problems and deciding on—and executing—the best solutions.

To ensure students have ample opportunity to develop these critical-thinking and decision-making skills, we believe business faculty should shift from teaching mostly principles and ideas to mostly applications and practices. And in doing so, they should emphasize the case method, which simulates real-world management challenges and opportunities for students.

To help educators facilitate this shift and help students get the most out of case-based learning, we have developed a framework for analyzing cases. We call it PACADI (Problem, Alternatives, Criteria, Analysis, Decision, Implementation); it can improve learning outcomes by helping students better solve and analyze business problems, make decisions, and develop and implement strategy. Here, we’ll explain why we developed this framework, how it works, and what makes it an effective learning tool.

The Case for Cases: Helping Students Think Critically

Business students must develop critical-thinking and analytical skills, which are essential to their ability to make good decisions in functional areas such as marketing, finance, operations, and information technology, as well as to understand the relationships among these functions. For example, the decisions a marketing manager must make include strategic planning (segments, products, and channels); execution (digital messaging, media, branding, budgets, and pricing); and operations (integrated communications and technologies), as well as how to implement decisions across functional areas.

Faculty can use many types of cases to help students develop these skills. These include the prototypical “paper cases”; live cases , which feature guest lecturers such as entrepreneurs or corporate leaders and on-site visits; and multimedia cases , which immerse students into real situations. Most cases feature an explicit or implicit decision that a protagonist—whether it is an individual, a group, or an organization—must make.

For students new to learning by the case method—and even for those with case experience—some common issues can emerge; these issues can sometimes be a barrier for educators looking to ensure the best possible outcomes in their case classrooms. Unsure of how to dig into case analysis on their own, students may turn to the internet or rely on former students for “answers” to assigned cases. Or, when assigned to provide answers to assignment questions in teams, students might take a divide-and-conquer approach but not take the time to regroup and provide answers that are consistent with one other.

To help address these issues, which we commonly experienced in our classes, we wanted to provide our students with a more structured approach for how they analyze cases—and to really think about making decisions from the protagonists’ point of view. We developed the PACADI framework to address this need.

PACADI: A Six-Step Decision-Making Approach

The PACADI framework is a six-step decision-making approach that can be used in lieu of traditional end-of-case questions. It offers a structured, integrated, and iterative process that requires students to analyze case information, apply business concepts to derive valuable insights, and develop recommendations based on these insights.

Prior to beginning a PACADI assessment, which we’ll outline here, students should first prepare a two-paragraph summary—a situation analysis—that highlights the key case facts. Then, we task students with providing a five-page PACADI case analysis (excluding appendices) based on the following six steps.

Step 1: Problem definition. What is the major challenge, problem, opportunity, or decision that has to be made? If there is more than one problem, choose the most important one. Often when solving the key problem, other issues will surface and be addressed. The problem statement may be framed as a question; for example, How can brand X improve market share among millennials in Canada? Usually the problem statement has to be re-written several times during the analysis of a case as students peel back the layers of symptoms or causation.

Step 2: Alternatives. Identify in detail the strategic alternatives to address the problem; three to five options generally work best. Alternatives should be mutually exclusive, realistic, creative, and feasible given the constraints of the situation. Doing nothing or delaying the decision to a later date are not considered acceptable alternatives.

Step 3: Criteria. What are the key decision criteria that will guide decision-making? In a marketing course, for example, these may include relevant marketing criteria such as segmentation, positioning, advertising and sales, distribution, and pricing. Financial criteria useful in evaluating the alternatives should be included—for example, income statement variables, customer lifetime value, payback, etc. Students must discuss their rationale for selecting the decision criteria and the weights and importance for each factor.

Step 4: Analysis. Provide an in-depth analysis of each alternative based on the criteria chosen in step three. Decision tables using criteria as columns and alternatives as rows can be helpful. The pros and cons of the various choices as well as the short- and long-term implications of each may be evaluated. Best, worst, and most likely scenarios can also be insightful.

Step 5: Decision. Students propose their solution to the problem. This decision is justified based on an in-depth analysis. Explain why the recommendation made is the best fit for the criteria.

Step 6: Implementation plan. Sound business decisions may fail due to poor execution. To enhance the likeliness of a successful project outcome, students describe the key steps (activities) to implement the recommendation, timetable, projected costs, expected competitive reaction, success metrics, and risks in the plan.

“Students note that using the PACADI framework yields ‘aha moments’—they learned something surprising in the case that led them to think differently about the problem and their proposed solution.”

PACADI’s Benefits: Meaningfully and Thoughtfully Applying Business Concepts

The PACADI framework covers all of the major elements of business decision-making, including implementation, which is often overlooked. By stepping through the whole framework, students apply relevant business concepts and solve management problems via a systematic, comprehensive approach; they’re far less likely to surface piecemeal responses.

As students explore each part of the framework, they may realize that they need to make changes to a previous step. For instance, when working on implementation, students may realize that the alternative they selected cannot be executed or will not be profitable, and thus need to rethink their decision. Or, they may discover that the criteria need to be revised since the list of decision factors they identified is incomplete (for example, the factors may explain key marketing concerns but fail to address relevant financial considerations) or is unrealistic (for example, they suggest a 25 percent increase in revenues without proposing an increased promotional budget).

In addition, the PACADI framework can be used alongside quantitative assignments, in-class exercises, and business and management simulations. The structured, multi-step decision framework encourages careful and sequential analysis to solve business problems. Incorporating PACADI as an overarching decision-making method across different projects will ultimately help students achieve desired learning outcomes. As a practical “beyond-the-classroom” tool, the PACADI framework is not a contrived course assignment; it reflects the decision-making approach that managers, executives, and entrepreneurs exercise daily. Case analysis introduces students to the real-world process of making business decisions quickly and correctly, often with limited information. This framework supplies an organized and disciplined process that students can readily defend in writing and in class discussions.

PACADI in Action: An Example

Here’s an example of how students used the PACADI framework for a recent case analysis on CVS, a large North American drugstore chain.

The CVS Prescription for Customer Value*

PACADI Stage

Summary Response

How should CVS Health evolve from the “drugstore of your neighborhood” to the “drugstore of your future”?

Alternatives

A1. Kaizen (continuous improvement)

A2. Product development

A3. Market development

A4. Personalization (micro-targeting)

Criteria (include weights)

C1. Customer value: service, quality, image, and price (40%)

C2. Customer obsession (20%)

C3. Growth through related businesses (20%)

C4. Customer retention and customer lifetime value (20%)

Each alternative was analyzed by each criterion using a Customer Value Assessment Tool

Alternative 4 (A4): Personalization was selected. This is operationalized via: segmentation—move toward segment-of-1 marketing; geodemographics and lifestyle emphasis; predictive data analysis; relationship marketing; people, principles, and supply chain management; and exceptional customer service.

Implementation

Partner with leading medical school

Curbside pick-up

Pet pharmacy

E-newsletter for customers and employees

Employee incentive program

CVS beauty days

Expand to Latin America and Caribbean

Healthier/happier corner

Holiday toy drives/community outreach

*Source: A. Weinstein, Y. Rodriguez, K. Sims, R. Vergara, “The CVS Prescription for Superior Customer Value—A Case Study,” Back to the Future: Revisiting the Foundations of Marketing from Society for Marketing Advances, West Palm Beach, FL (November 2, 2018).

Results of Using the PACADI Framework

When faculty members at our respective institutions at Nova Southeastern University (NSU) and the University of North Carolina Wilmington have used the PACADI framework, our classes have been more structured and engaging. Students vigorously debate each element of their decision and note that this framework yields an “aha moment”—they learned something surprising in the case that led them to think differently about the problem and their proposed solution.

These lively discussions enhance individual and collective learning. As one external metric of this improvement, we have observed a 2.5 percent increase in student case grade performance at NSU since this framework was introduced.

Tips to Get Started

The PACADI approach works well in in-person, online, and hybrid courses. This is particularly important as more universities have moved to remote learning options. Because students have varied educational and cultural backgrounds, work experience, and familiarity with case analysis, we recommend that faculty members have students work on their first case using this new framework in small teams (two or three students). Additional analyses should then be solo efforts.

To use PACADI effectively in your classroom, we suggest the following:

Advise your students that your course will stress critical thinking and decision-making skills, not just course concepts and theory.

Use a varied mix of case studies. As marketing professors, we often address consumer and business markets; goods, services, and digital commerce; domestic and global business; and small and large companies in a single MBA course.

As a starting point, provide a short explanation (about 20 to 30 minutes) of the PACADI framework with a focus on the conceptual elements. You can deliver this face to face or through videoconferencing.

Give students an opportunity to practice the case analysis methodology via an ungraded sample case study. Designate groups of five to seven students to discuss the case and the six steps in breakout sessions (in class or via Zoom).

Ensure case analyses are weighted heavily as a grading component. We suggest 30–50 percent of the overall course grade.

Once cases are graded, debrief with the class on what they did right and areas needing improvement (30- to 40-minute in-person or Zoom session).

Encourage faculty teams that teach common courses to build appropriate instructional materials, grading rubrics, videos, sample cases, and teaching notes.

When selecting case studies, we have found that the best ones for PACADI analyses are about 15 pages long and revolve around a focal management decision. This length provides adequate depth yet is not protracted. Some of our tested and favorite marketing cases include Brand W , Hubspot , Kraft Foods Canada , TRSB(A) , and Whiskey & Cheddar .

Art Weinstein

Art Weinstein , Ph.D., is a professor of marketing at Nova Southeastern University, Fort Lauderdale, Florida. He has published more than 80 scholarly articles and papers and eight books on customer-focused marketing strategy. His latest book is Superior Customer Value—Finding and Keeping Customers in the Now Economy . Dr. Weinstein has consulted for many leading technology and service companies.

Herbert V. Brotspies

Herbert V. Brotspies , D.B.A., is an adjunct professor of marketing at Nova Southeastern University. He has over 30 years’ experience as a vice president in marketing, strategic planning, and acquisitions for Fortune 50 consumer products companies working in the United States and internationally. His research interests include return on marketing investment, consumer behavior, business-to-business strategy, and strategic planning.

John T. Gironda

John T. Gironda , Ph.D., is an assistant professor of marketing at the University of North Carolina Wilmington. His research has been published in Industrial Marketing Management, Psychology & Marketing , and Journal of Marketing Management . He has also presented at major marketing conferences including the American Marketing Association, Academy of Marketing Science, and Society for Marketing Advances.

Related Articles

CASE TEACHING

We use cookies to understand how you use our site and to improve your experience, including personalizing content. Learn More . By continuing to use our site, you accept our use of cookies and revised Privacy Policy .

case study discussion rubric

Case Study - Rubric

  • Available Topics
  • Top Documents
  • Recently Updated
  • Internal KB

CTLM Instructional Resources

This KB document is part of a larger collection of documents on Equity and inclusion. More Equity & Inclusion documents

Rubric example: a case study

  • Investigation and Research Discussions
  • Case Study - Description
  • Case Study - Example
  • Affordances of Online Discussions
  • Steps for Building an Online Asynchronous Discussion
  • Using Online Asynchronous Discussions to Increase Student Engagement & Active Learning

case study discussion rubric

Case Studies

Case studies (also called "case histories") are descriptions of real situations that provide a context for engineers and others to explore decision-making in the face of socio-technical issues, such as environmental, political, and ethical issues. case studies typically involve complex issues where there is no single correct answer--a student analyzing a case study may be asked to select the "best" answer given the situation 1 . a case study is not a demonstration of a valid or "best" decision or solution. on the contrary, unsuccessful or incomplete attempts at a solution are often included in the written account. 2.

The process of analyzing a case study encourages several learning tasks:

Exploring the nature of a problem and circumstances that affect a decision or solution

Learning about others' viewpoints and how they may be taken into account

Learning about one's own viewpoint

Defining one's own priorities

Making one's own decisions to solve a problem

Predicting outcomes and consequences 1

Student Learning Outcomes in Ethics

Most engineering case studies available pertain to engineering ethics. After a two year study of education in ethics sponsored by the Hastings Center, an interdisciplinary group agreed on five main outcomes for student learning in ethics:

Sensitivity to ethical issues, sometimes called "developing a moral imagination," or the awareness of the needs of others and that there is an ethical point of view;

Recognition of ethical issues or the ability to see the ethical implications of specific situations and choices;

Ability to analyze and critically evaluate ethical dilemmas, including an understanding of competing values, and the ability to scrutinize options for resolution;

Ethical responsibility, or the ability to make a decision and take action;

Tolerance for ambiguity, or the recognition that there may be no single ideal solution to ethically problematic situations 2 .

These outcomes would make an excellent list of attributes for designing a rubric for a case analysis.

Ideas for Case Study Assignments

To assign a case analysis, an instructor needs

skill in analyzing a case (and the ability to model that process for students)

skill in managing classroom discussion of a case

a case study

a specific assignment that will guide students' case analyses, and

a rubric for scoring students' case analyses.

Below are ideas for each of these five aspects of teaching with case studies. Another viewpoint is to consider how not to teach a case study .

1. Skill in analyzing a case

For many engineering instructors, analyzing cases is unfamiliar. Examining completed case analyses could help develop case analysis skills. As an exercise for building skill in analyzing cases, use the generic guidelines for case analysis assignments (#4 below) to carefully review some completed case analyses. A few completed case analyses are available:

Five example analyses of an engineering case study

Case study part 1 [Unger, S. The BART case: ethics and the employed engineer. IEEE CSIT Newsletter. September 1973 Issue 4, p 6.]

Case study part 2 [Friedlander, G. The case of the three engineers vs. BART. IEEE Spectrum. October 1974, p. 69-76.]

Case study part 3 [Friedlander, G. Bigger Bugs in BART? IEEE Spectrum . March 1973. p32,35,37.]

Case study with an example analysis

2. Skill in managing classroom discussion of a case

Managing classroom discussion of a case study requires planning.

Suggestions for using engineering cases in the classroom

Guidelines for leading classroom discussion of case studies

3. Case studies

Case studies should be complex enough and realistic enough to be challenging, yet be manageable within the time frame. It is time-consuming to create case studies, but there are a large number of engineering case studies online.

Online Case Libraries

Case Studies in Technology, Ethics, Environment, and Public Policy

Teaching Engineering Ethics: A Case Study Approach

The Online Ethics Center for Engineering and Science

Ethics Cases

The Engineering Case Library

Cases and Teaching Tips

4. A specific assignment that will guide students' case analyses

There are several types of case study assignment:

Nine approaches to using case studies for teaching

Written Case Analysis

Case Discussions

Case analyses typically include answering questions such as:

What kinds of problems are inherent in the situation?

Describe the socio-technical situation sufficiently to enable listeners (or readers) to understand the situation faced by the central character in the case.

Identify and characterize the issue or conflict central to the situation. Identify the parties involved in the situation. Describe the origins, structure, and trajectory of the conflict.

Evaluate the strengths and weaknesses of the arguments made by each party.

How would these problems affect the outcomes of the situation?

Describe the possible actions that could have been taken by the central character in the case.

Describe, for each possible action, what the potential outcomes might be for each party involved.

Describe what action was actually taken and the outcomes for each party involved.

How would you solve these problems? Why?

Describe the action you would take if you were the central character in the case. Explain why.

What should the central character in the situation do? Why?

Describe the action you think that the central character in the case should take. Explain why.

What can be learned from this case?

Delineate the lessons about ethical (or other) issues in engineering that are illuminated by this case.

This list is adapted from two online case analysis assignments by McGinn from 3 & 4 ):

5. A rubric for scoring students' case analyses

Case studies help students explore decision-making in the face of issues. Thus, for an engineering ethics case study, the outcomes that can be assessed by scoring case analyses are a) sensitivity to ethical issues, b) recognition of ethical issues, c) the ability to analyze and critically evaluate ethical dilemmas, d) the ability to make an ethical decision and take action, and e) tolerance for ambiguity. Scoring rubrics for ethics case analyses should address these outcomes, not basic knowledge of the ethical standards of the profession. Professional standards can best be assessed by a traditional graded exam in which students must demonstrate, for example, which practices are ethically acceptable versus which are in violation of ethical standards given a hypothetical scenario 5 .

Making Scoring/Grading Useful for Assessment

General principles for making scoring/grading useful for assessment ( rubrics )

Example rubrics

For a written analysis of a case study in engineering

For a written analysis of a case study in general #1

For a written analysis of a case study in general #2

For a written and oral analysis of a case study by a group

For an oral analysis of a case study by a group

For a written analysis of a case study on ethics

For a self-assessment of learning from a case study

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Advanced Search
  • Journal List
  • Am J Pharm Educ
  • v.72(3); 2008 Jun 15

Multiple Rubric-based Assessments of Student Case Presentations

To evaluate a rubric-based method of assessing pharmacy students' case presentations in the recitation component of a therapeutics course.

A rubric was developed to assess knowledge, skills, and professional behavior. The rubric was used for instructor, student peer, and student self-assessment of case presentations. Rubric-based composite scores were compared to the previous dichotomous checklist-based scores.

Rubric-based instructor scores were significantly lower and had a broader score distribution than those resulting from the checklist method. Spring 2007 rubric-based composite scores from instructors and peers were significantly lower than those from the pilot study results, but self-assessment composite scores were not significantly different.

Conclusions

Successful development and implementation of a grading rubric facilitated evaluation of knowledge, skills, and professional behavior from the viewpoints of instructor, peer, and self in a didactic course.

INTRODUCTION

The Accreditation Council for Pharmacy Education (ACPE) Standards 2007 encourage the multidimensional assessment of knowledge, skills, and behaviors. 1 While knowledge and skills are the focus of our assessment efforts in the didactic coursework, the evaluation of professional behavior has been lacking. The recitation component of a Therapeutics I course offered an ideal environment for formally assessing each of the 3 dimensions because of its small group setting and emphasis on the application of material presented in the didactic component of the course. A method of multidimensional assessment was developed that would provide students with an opportunity to reflect upon and provide feedback regarding their own performances and that of their peers.

We describe a pilot study and implementation of a grading rubric that facilitates assessment of student case presentations from 3 sources: instructors, peers, and self. The objectives of the pilot study were to (1) determine the difference between checklist and rubric-based instructor scores; (2) determine the difference between composite instructor, peer, and self-assessments using the rubric; and (3) determine the difference between instructor, peer, and self-assessments for each dimension (knowledge, skills, and behavior) of the rubric. We also describe our experience with implementation of the rubric as the formal grading method in the subsequent academic year.

Approval was obtained from the Institutional Review Board (IRB) at the University of Arkansas for Medical Sciences to conduct the study. A waiver of written informed consent was granted and students indicated consent by participating in the study. All second year (P2) pharmacy students who were enrolled in the spring 2006 Therapeutics I course were invited to participate.

Recitation is a weekly 2-hour practical case-based portion of the Therapeutics I course for P2 pharmacy students at the University of Arkansas for Medical Sciences College of Pharmacy, Little Rock. It is facilitated by pharmacy residents and fellows who have clinical instructor appointments with the College of Pharmacy. In spring 2006, students were assigned to 1 of 6 sections, which were further divided into 4 small groups of 3-4 students each. During the first hour of recitation, students worked in small groups to review therapy recommendations for the case and prepare presentations; instructors provided guidance during this hour. All students received the same 4 cases each week and each was expected to prepare for all cases before the recitation session. Cases closely parallel the lectures presented in the didactic portion of the course. The second hour was devoted to case presentations, with 1 student per small group presenting a patient case. As there were 4 small groups per section, each small group presented a different case. After each case presentation, the instructor asked a series of questions to further measure the student's knowledge base and ability to defend the recommendations. Other small group members were encouraged to answer questions if the presenter was having difficulty, but the case presentation grade was assigned to the presenter only. Prior to this study, the presentations were graded solely by the instructor with a case-specific dichotomous checklist that assessed the student's therapeutic recommendations and presentation style (Figure ​ (Figure1). 1 ). Instructors were trained in this grading method and overseen by the course coordinator. The checklist grade was calculated by the percentage of items that received a “pass,” resulting in possible scores ranging from 0 to 100%. This was sometimes accompanied by constructive criticism offered by the instructor.

An external file that holds a picture, illustration, etc.
Object name is ajpe58fig1.jpg

Example checklist used to assess pharmacy students' case presentations.

For the purpose of this pilot study, a rubric was developed by the investigators to assess 3 dimensions of performance: knowledge, skills, and professional behavior. When determining which dimensions should be included and the content for each, we sought to mirror the recommendations from both the ACPE Standards 2007 1 and the Center for the Advancement of Pharmaceutical Education (CAPE) Educational Outcomes 2 to ensure content validity. The resulting rubric was reviewed and revised by the investigators multiple times, and feedback was provided by non-investigator instructors. The knowledge dimension of the rubric contained 2 assessment items and the skills and behavior dimensions each contained 4 assessment items. Three levels of competency were described for each assessment item (Figure ​ (Figure2), 2 ), and instructors and students were asked to circle the description that best represented each presenter's performance. A numerical composite score was calculated from each completed rubric by using the following conversion for each item: lowest level of performance = 5 points, middle level of performance = 7.5 points, and highest level of performance = 10 points. Possible scores ranged from 50 to 100 points. Scores for the knowledge, skills, and behavior dimensions were calculated by adding only the points for the items within each dimension. Students and instructors were not informed of this conversion method.

An external file that holds a picture, illustration, etc.
Object name is ajpe58fig2.jpg

Rubric for assessing pharmacy students' case presentations based on knowledge, skills and behaviors.

Students and recitation instructors were educated regarding the application of the rubric by an investigator. The different dimensions of the rubric were explained and students and instructors were given the opportunity to ask questions. Further clarification was provided during the semester when requested. The rubric was used for instructor, peer, and self-assessments of all presentations for the pilot study. Peers were defined as the non-presenting members of each recitation section (13-15 students). All students participated each week by completing self-assessments when presenting and peer assessments when not presenting. Instructors completed the dichotomous checklist and rubric for each student presentation. Students and instructors had several minutes immediately following each presentation to complete the assessments. During the pilot study, the dichotomous checklist used by instructors continued to be the formal grading method for all presentations.

Rubric-based composite scores were calculated for instructor, peer, and self-assessments. Instructor composite scores resulting from the rubric-based method were compared to the dichotomous checklist scores using a 2-tailed t test. Rubric-based composite scores for instructor, peer, and self-assessments were compared using a one-way analysis of variance (ANOVA). Scores for individual dimensions (knowledge, skills, behavior) were calculated for instructor, peer, and self-assessments using ANOVA. P values less than 0.05 were considered significant in all analyses.

Following the results of the pilot study, the grading methodology for Therapeutics I recitation was redesigned, with the rubric-based method of assessment used as the sole grading mechanism for case presentations. In spring 2007, students and instructors were oriented to the rubric in the same manner as described in the methods for the pilot study. Implementation of the rubric-based assessments during spring 2007 incorporated a few changes from the methods described in the pilot study. First, only the 3-4 students within the presenter's small group were asked to provide a peer assessment of the case presentation. This decision was made after reviewing the peer assessments from the pilot study, where only 43.7% of the peer assessment rubrics were fully completed. Further review revealed that more rubrics were completed by student peers within the small group than by other student peers outside the small group, indicating that these students were more comfortable assessing their small group peers' presentations. Second, students completed the rubric-based self- and peer assessments online using WebCT rather than using a paper rubric, thus allowing more time for reflection and assessment. Students and instructors had up to 1 week following the presentation to complete the assessments. The self-assessment comprised 25% of the student's final presentation score, and instructor assessments comprised the remaining 75% of the score. Although peer assessment scores were not used in calculating the presentation scores, students were required to complete peer assessments to receive participation credit in recitation. Comments from the peer assessments were used to supply additional feedback for improvement.

IRB approval was obtained to compare scores from Spring 2007 to those from the pilot study. Rubric-based composite instructor scores from Spring 2007 and the pilot study were compared using a 2-tailed t test. Scores generated from peer and self-assessments were analyzed in the same manner. Additionally, rubric-based composite scores for instructor, peer, and self-assessments during the spring 2007 implementation were compared using a one-way ANOVA. P values less than 0.05 were considered significant in all analyses.

During the pilot study, 86 students were enrolled in Therapeutics I . Each student presented 3-4 case presentations (mean 3.3) during the semester, and each presentation was assessed using the rubric and checklist methods. Incomplete rubrics were not included in the analysis. The percentage of complete rubrics varied between evaluators, with 89.8%, 43.7%, and 97.9% of instructor, peer, and self-assessments completed. The mean instructor rubric-based composite score was significantly lower than the mean checklist-based score (94.0 ± 5.7 versus 97.7 ± 4.9, respectively, p < 0.001). Figure ​ Figure3 3 demonstrates that the rubric method produced a broader distribution of scores than the checklist method. Rubric-based composite peer assessment scores were statistically higher than rubric-based instructor or self-assessment scores ( p < 0.001 for each comparison), but there was no difference between instructor and self-assessment composite scores ( p = 0.54, Table ​ Table1 1 ).

An external file that holds a picture, illustration, etc.
Object name is ajpe58fig3.jpg

Frequency distributions of rubric and checklist-based recitation scores from instructors.

Rubric-based Scores for Instructor, Peer, and Self-Assessments a

An external file that holds a picture, illustration, etc.
Object name is ajpe58tbl1.jpg

When scores for knowledge, skills, and behavior were analyzed (Table ​ (Table1), 1 ), instructor scores were significantly lower for knowledge and higher for behavior than self-assessment scores ( p < 0.001 for each comparison). Scores for skills were not different between instructor and self-assessments ( P > 0.10). Peer assessment scores were significantly higher for all 3 rubric dimensions than instructor or self-assessment scores ( p < 0.01 for each comparison).

When recitation scores from the spring 2007 semester of Therapeutics I were analyzed (Table ​ (Table2), 2 ), rubric-based composite scores from instructors and peers were found to be significantly lower compared to pilot study results ( p = 0.01 and p < 0.001, respectively), but self-assessment composite scores were not significantly different ( p = 0.06). Spring 2007 instructor, peer, and self-assessment scores were compared, and the instructor composite score was significantly lower than both peer assessment and self-assessment composite scores ( p < 0.001 for each comparison). Peer and self-assessment composite scores were not significantly different ( p = 0.18).

Comparison of Pilot Study Rubric Composite Scores to Spring 2007 Rubric Composite Scores a

An external file that holds a picture, illustration, etc.
Object name is ajpe58tbl2.jpg

In this study, rubric-based composite scores provided a broader distribution of student presentation scores than the checklist-based method. One explanation for this increase in score distribution may be that the checklist-based method mainly assessed knowledge recall and application, which are among the lower levels of Bloom's Taxonomy, 3 while the rubric-based method allows for the assessment of higher-level educational goals such as synthesis and evaluation. Thus, the rubric-based method provides a more global assessment of performance than the checklist-based method because it allows for assessment of critical thinking and professional behavior in addition to knowledge and presentation skills. The rubric specifically evaluates professional attire, interactions with small group peers and instructors, approach to group participation, and preparedness for case presentations. This rubric is valuable in providing an assessment of professional behavior in the didactic coursework, allowing feedback to students who might not otherwise receive it.

The results from spring 2007, although similar to the pilot study, had important differences. For example, the peer assessment scores for the spring 2007 semester were lower than peer assessment scores from the pilot study. This difference may have resulted from the provision of peer assessments by students within the presenter's small group and not from the entire recitation section. This was expected because the procedural change resulted in peer assessments from students with direct interaction versus those peripheral to the small group. The mean difference in instructor assessments, although small, was statistically significant between the spring 2007 and pilot study results. This is surprising, since the spring 2007 rubric-based scores were used as the official grades for students, whereas the pilot study scores did not impact students' grades. Notably, the spring 2007 self-assessment scores did not differ significantly from those in the pilot study. When the rubric-based method was formally implemented in spring 2007, the contribution of students' self-assessments carried an inherent risk of score inflation. However, this risk may have been minimized by limiting the contribution of the self-assessments to only 25% of the presentation scores.

The conversion to a numerical score introduces a possible limitation. The method of conversion was based on the logic that the highest level of performance deserved full credit, middle level of performance deserved the equivalent of a letter grade of C, and the lowest level of performance deserved a failing grade, but not a zero. As a result, the possible rubric-based scores ranged from 50-100 points. Although the rubric was converted to a numerical score, its use resulted in a broader distribution of grades than the checklist method.

This rubric allows facilitation of assessment from 3 sources: instructors, peers, and self. The importance of self-assessment exercises for students in health care professions is widely recognized, as evidenced by their inclusion in accreditation standards for colleges of pharmacy 1 as well as colleges of medicine. 4 Most research on student self-assessments in the health sciences has been conducted among medical students, and the data are conflicting regarding the accuracy of the self-assessment. 5 - 10 Student self-assessment accuracy is most commonly measured by comparison to faculty assessments and differences are assumed to be indicative of inaccurate student self-assessments resulting from lack of clear assessment method or guidelines, limited self-assessment experience, or assessment of effort rather than performance. 5 - 10 Interestingly, self-assessments are most valuable when accompanied by assessments from instructors and/or peers. 11

In the literature, peer assessment is usually conducted anonymously and generally includes assessment of observed tasks as well as professionalism. Several studies have shown that peer assessment is accepted and valued by students and residents. 12 - 14 However, other studies show that students have concerns about its validity 15 and its contribution to course grades. 16 An expected additional benefit of the peer assessment process is that as students assess each other, they may gain valuable experience in the process of assessment. 17 The ideal situation would be one which combines instructor, peer, and self-assessments for the same observed task. These multiple points of view are useful because peers observe a different set of skills than do instructors. 18 It also allows for the comparison of self-assessments to assessments from others, thereby enabling continued improvement of self-assessment skills. 8 , 11

Summarized in Table ​ Table3 3 are several studies addressing the assessment of pharmacy, nursing, or medical students in problem-based learning courses or courses with group projects. 7 , 9 , 14 , 16 , 19 - 27 Our study is similar to these studies in that student assessments were of a task that required the practical application of knowledge. While most of these studies utilized peer and/or self-assessments in addition to instructor assessments, only a few comprehensively evaluated knowledge, skills, and professional behavior. Our method combines assessment of knowledge, skills, and professional behavior from all 3 sources: instructors, peers, and self. It is novel because it uses a rubric, providing both presenters and evaluators with specific expectations for performance.

Literature Summary for Student Assessment in Problem-Based Learning or Group Projects

An external file that holds a picture, illustration, etc.
Object name is ajpe58tbl3.jpg

a Problem-based learning

b Likert scale in which the middle of the scale represents the best performance and the ends of the scale represent opposite extremes

c Written descriptions of the lowest, middle, and highest numbers on the Likert scale

CONCLUSIONS

This study describes the successful development and implementation of a grading rubric that facilitates the evaluation of knowledge, skills, and professional behavior from the viewpoints of instructor, peer, and self-assessments in a didactic course. The rubric serves as a guide for students to appreciate the desired performance outcomes and to facilitate the provision of constructive feedback. The use of this rubric was the first step in attempting to achieve a recitation grading mechanism that complemented the knowledge and skill assessment achieved by the examinations in the didactic portion of the Therapeutics I course.

ACKNOWLEDGMENTS

The authors wish to thank Kimberley Dickey, PharmD, for her contributions during the initiation of this project and data collection.

case study discussion rubric

Teaching Online Pedagogical Repository

Create a Case Method Group Activity to Engage Students in Critical Thinking

Tags: Adult Learning , Assessment , Assignments , Blog , Case method , Cognitive Theory , Collaborative Learning , Community of Inquiry , Critical Thinking , Discussions , Experiential Learning , Google , Learning Activities , LMS , Problem-Based Learning , Scaffolding , Social Media , Video , Wiki

Description

The case method group activity is an instructional design strategy that involves faculty members providing one or more case studies to which groups of students respond. The case(s) could be a real-life case or simulation. It could be description of key concept(s) applied, a story or scenario, an actual case study, a problem or mystery, a performance, a visual, or an example.

The case method in online learning as an intervention presents students with ill-structured, real-world derived problems with multiple solutions (Choi & Lee, 2009). In a group activity this case method has the potential to harnesses the effectiveness of collaborative learning (Kolb, 1984) and group activities provide a space for collaborative problem solving, fostering a constructivist learning environment with potential to build a community of learning (Jonassen, 1997). The teaching facilitator can influence learners’ engagement in and adoption of the activity by communicating the relative advantage of key features of the online environment (Karamanos & Gibbs, 2012), as well as mapping and intervening in the group interactions to keep students focused on the problem (Etmer & Koehler, 2014). This mapping creates a plan for instructors to scaffold (or build in techniques to progressively support students to greater levels of learning independence and effective group interactions). Introduction of scaffolds and learning resources, perhaps additional readings and activities, presented at later stages of problem solving were associated with deep meaningful learning and critical thinking (Choi & Lee, 2009).

Because the case reflects a real-life situation, as the group members interact with each other, they should uncover multiple solutions, perspectives, or methods of analyzing the situation, with no single right answer. This divergence is important to encourage for fostering deeper levels of learning and critical thinking (Choi & Lee, 2009). A guiding question for the lesson can offer some parameters for faculty to map and scaffold activities, guiding students’ interactions as they engage with others in their groups about the case. This guiding question should depend on the purpose of the instruction (University of Illinois, 2015). The process of a faculty member scaffolding activities should result in more than one individual or group deliverable associated with it and a corresponding timeline for each. Consider, too, whether each component will have a group or individual grade (Carnegie Mellon University, 2015).

Link to example artifact(s)

As an example of the case method group activity, a faculty member teaching an industrial/organizational psychology course divided the students into groups based on time zones and created a discussion forum for each group. They completed a learning team charter to establish their group covenant. The parameters for the group activity were well-defined: students completed a group charter to agree on expectations for each member’s contribution to the group, the faculty provided an explanation of the purpose of the group activity for learning the material and succeeding in the course. The faculty explicitly stated how the quality of interactions and ideas derived from the group conversation and supported with evidence could contribute to the success of individual assignments.

The faculty provided the same case scenario across each group discussion forum. The case described in writing a company whose strict hierarchy and “us” (management) versus “them” (workers) mentality had led to a dysfunctional workplace environment with punitive acts from management and passive aggressive behaviors from employees. The faculty prompted students to analyze the situation using management and leadership approaches and theories from the course as a group by midway through the course, brainstorm as a group solutions to remedy the situation by applying key course concepts from middle to the end of the course, and submit an individual solution to the problem the case presented as the final assignment.

The faculty member interacted with each group several times throughout the course in their discussion forums, guiding them to consider important motivational and management theories (like Maslow’s hierarchy of needs and McGregor’s theory x and y) to analyze the behavioral dynamics of management and the employees in the case. Additionally, a rubric was associated with each component presented prior to the assignment to set expectations and utilized by the faculty member for grading. To assess critical thinking, elements from the AASU Value rubric were incorporated into the individual solution activity rubric (2017). Students were graded individually even though they were interacting as a group for their mid-course analysis of the scenario, as a group on the solution brainstorming activity for a group grade, and individually on their submissions of their individual solution to the case.

Students’ critical thinking improvements and favorable reaction provided good evidence for the success of the case scenario activities in this course. During the discussion, students often related the scenario to their own real-life experiences in workplace settings. As the discussions progressed, students began identifying these experiences with key concepts, referring and citing course content, and this habit transferred to their individual solutions to the case presented in their papers. Some students developed a clear thesis for their perspectives, recounting nuances of the situation in the scenario (such as the organizational structure and emerging management styles) to substantiate their position. Even fewer students gave multiple solutions and explained why one is better than another. Students generally responded positively to the course and case scenario format. They indicated that they appreciated the real-life examples from other students and expressed that the group discussion contributed development of their individual submissions. Satisfaction with the course, as indicated on students’ end-of-course reviews, was high to very high.

Applying this case method group activity strategy to other disciplines should result in similar success, strengthening students’ critical thinking skills. This strategy is definitely generalizable, as the aim is for students’ collaboration for achieving the course or module objective(s) associated with the activity/assignment (University of Illinois, 2015), guiding students to:

• Identify key concepts reflected by a case, • Situate a case within a given system, • Summarize or recapitulate a case, • Generalize patterns or symbolic representations within a case, • Generate plausible causes that result in a case, • Analyze the components of a case, • Assess or judge the appropriate application presented in a case scenario, • Solve a problem that the case presents or that the faculty presents about the case Any discipline where students would benefit cognitively from collaboration to achieve one of the above objectives could apply this strategy: create a case method group activity to engage students in critical thinking.

Individual Paper Case Study Rubric

Case Study Rubric for Group Collaboration Discussion

Instructor: Revathi Viswanathan Course: Biotechnology

Students were asked to discuss case studies relating to their subject, and Edmodo was used as a learning platform for handling them. The purpose of integrating the technological tool was to encourage students to actively participate in the teaching and learning process even beyond their classroom. Besides this, Edmodo, as an application could be accessed both in a computer and a mobile, which in turn helps teachers to post resources, initiate discussions, create small groups, and enroll students to do collaborative tasks.

As part of the classroom based research, the students were put in small groups, and inputs for case study was posted as video files and reading texts to each group in the Edmodo page. Each group was given guidance on holding case study discussions. The preliminary discussion was initiated through brainstorming questions, which encouraged students to get to the important issue or aspect of the case study. For example, the following questions were posted related to the case study on ‘ Regenerative medicine- Current therapies and future directions’:

  • What do you know about regenerative medicine?
  • How does it help common man?

Similarly, one of the groups discussed ‘Genetic Engineering of Animals: ethical issues, including welfare concerns’ and the following questions were asked:

  • Discuss how ‘deletion of genes, or the manipulation of genes already present’ affects the animals.
  • What do you think about this as a biotechnology student?

After a few dialogues online, it was felt that most of the students could not identify the main aspect of the case study and the teacher researcher had to intervene by posting guidelines for discussing a case study (Source: https://plato.acadiau.ca/courses/Busi/IntroBus/CASEMETHOD.html#CASEMETHOD ). They were told about the process by which a case study has to be analysed. They were asked to look for issues that are stated in the case study, critically read and see how the issue is handled in depth. Then, they had to look for opening paragraph, background information, specific area of interest covered, specific problem stated, alternatives given and the conclusion drawn from the discussion, in the case study.

Besides posting these tips for handling case studies, the teacher explained the components of a case study (stated above) in the class. It was felt that this online collaborative activity had to be handled by following the online collaborative theory advocated by Harasim (2012). According to her, a teacher plays an important role (in an online collaborative learning scenario) in the process of knowledge construction among students, by providing inputs and integrating the core concept along with the subject domain. In this context, it must be stated that the teacher researcher had already brought in the integration of biotechnology related case studies for discussion. However, considering the extent to which they could use the subject knowledge for discussing the given case study, it was evident that the students expected teacher’s intervention. In other words, the teacher had to draw their attention to the main issue of the case study by posting a few specific (case study related) questions.

For example, the group which was working on the case study, ‘Genetic engineering of animals’, were asked to focus on the specific concepts. The following question was posted in Edmodo group page:

  • How does it affect an animal when it is genetically modified or genetically altered or genetically manipulated or transgenic, and biotechnology-derived
  • How will the animal cope with when it is modified?

The extent to which the students of respective groups (group A & B) could discuss the case study by using their subject knowledge was evaluated by comparing two groups (both before-the-intervention and after-the-intervention of the teacher) using Causal-Comparative method. The analysis of performance of the group members was done with the help of the rubrics, ‘Undergraduate Case Analysis Rubrics’ (Source https://www.onlineethics.org/File.aspx?id=31203&v=859a7ffb ). The frequency polygon drawn for both groups (Figures 1 and 2) and the ANOVA test scores evaluated showed variation, particularly in group A’s performance (ie before and after teacher’s intervention).

Frequency polygon for the initial performance

It was evident that students’ application of subject knowledge in their discussion had promoted communicative ability. Further, it proved the application of online collaborative theory in encouraging students to contribute to online discussions.

Link to scholarly references

Association of American Colleges & Universities (2017). Critical thinking VALUE rubric. Retrieved from  https://www.aacu.org/value/rubrics/critical-thinking

Carnegie Mellon University Eberly Center (2015). How can I assess group work? Retrieved from  https://www.cmu.edu/teaching/designteach/design/instructionalstrategies/groupprojects/assess.html

Choi, I., & Lee, K. (2009). Designing and implementing a case-based learning environment for enhancing ill-structured problem solving: Classroom management problems for prospective teachers. Educational Technology Research and Development , 57( 1 ), 99-129.  https://doi.org/10.1007/s11423-008-9089-2

David, H. J. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology Research and Development , 45( 1 ). Retrieved from  http://www.webkelley.com/HBS/ID%20Models%20for%20Well-Structured.pdf

Ertmer, P. A., & Koehler, A. A. (2014). Online case-based discussions: Examining coverage of the afforded problem space. Educational Technology Research and Development , 62( 5 ), 617-636.  https://doi.org/10.1007/s11423-014-9350-9

Jonassen, D. H. (1999). Designing constructivist learning environments. In C. M. Reigeluth I nstructional-design theories and models: Volume II (pp. 215-239). Mahwah, N.J: Lawrence Erlbaum Associates. Retrieved from  https://www.savoiabenincasa.gov.it/wp-content/uploads/2016/04/1999-Jonassen.pdf

Karamanos, N., & Gibbs, P. (2012). A model for student adoption of online interactivity. Research in Post-Compulsory Education , 17( 3 ), 321-334.  https://doi.org/10.1080/13596748.2012.700108

Kolb, D. (1984). Experiential learning: Experience as the source of learning and development . Englewood Cliffs, NJ: Prentice Hall. Retrieved from  https://www.pearson.com/us/higher-education/program/Kolb-Experiential-Learning-Experience-as-the-Source-of-Learning-and-Development-2nd-Edition/PGM183903.html

University of Illinois. (2015). Online teaching activity index: Case study or case based index. Retrieved from  www.ion.uillinois.edu%2Fresources%2Fotai%2Fcasestudies.asp&token=yz%2BG1QALcwhrBLaVIIOV1qkwVJCS27mZAH624RoGdAc%3D

Post Revisions:

  • August 7, 2020 @ 20:33:26 [Current Revision]
  • August 7, 2020 @ 20:33:26
  • October 2, 2019 @ 14:57:03
  • September 30, 2019 @ 18:49:58
  • September 30, 2019 @ 18:46:31
  • help_outline help

iRubric: Case Study rubric

  • Case study rubric

case study discussion rubric

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

  • We're Hiring!
  • Help Center

paper cover thumbnail

Case Study Discussion Grading Rubric Guidelines & Scoring Rubric Chamberlain College of Nursing NR 535Theoretical Foundations and Instructional Strategies for the Nurse Educator

Profile image of Homework Minutes

Nursing Case Study Help

Purpose The purpose of this assignment is to provide an opportunity for learners to: (a) articular key questions to be asked in a grievance; (b) review an academic integrity policy; (c) determine a decision in a grievance process. Course Outcomes: CO 5. Examine professional ethical legal values in the role of the nurse educator (PO 4) Due Date: Sunday 11:59 p.m. MT at the end of Week 7 Total Points Possible: 100 Requirements: Week 7: Case Study Part I The case study has three distinct parts. Each part of the case study will open and close on specific days of the week. Part I of the case study opens on Sunday and will close on Wednesday of the week. You cannot add additional information to Part I once it closes on Wednesday. In this part of the case study you are to identify a 2 key questions for the student relevant to the grievance and 2 key questions for the faculty member. Include the rational for these questions.

Related Papers

Nursing Ethics

Marsha Fowler

case study discussion rubric

Ora Strickland

Journal of Professional Nursing

Mary Ellen Smith Glasgow

The Journal of School Nursing

Marjorie Schaffer

This study explored school nurses’ experience of ethical conflict in school nursing through interviews with six school nurses. The study examined how school nurses resolved ethical problems and the rationale used to resolve them. Emergent themes of ethical problems were professional relationship conflicts, delegation to and supervision of health assistants, child protection reporting, maintaining confidentiality, Do Not Resuscitate policy, and pressure to work outside of nursing practice standards. School nurses did not use ethical decision-making models in resolving conflict but demonstrated the use of professional standards, ethical principles, and personal values as rationale to resolve ethical problems. Results of this study suggested that school nurses would benefit from additional knowledge about ethical decision-making models. School nurses would also profit from hearing each other’s voices through dialogue about ethical problems and decision making.

Nurse Education Today

Paul Snelling

Despite nursing’s move into higher education, academic freedom has received little attention within the literature. After discussing the concept of academic freedom, this paper argues that there is a potential tension between academic freedom and the requirement to educate student nurses who are fit for practice. One way in which this tension might be revealed is in the marking of student assignments. We ask the question – how should nurse educators mark an essay which is sufficiently analytical but reaches moral conclusions that lie outside the Code of Professional Conduct? We argue that despite an understandable temptation to penalise such an essay, invoking the Code of Professional Conduct to do so, no penalty should be applied, and academic freedom for students within higher education should be encouraged. This is because first, academic freedom is a good in itself especially as it allows unconventional and unpalatable conclusions to be discussed and rebutted, and second, applying a penalty on these grounds is necessarily inconsistent

Nurse education today

Aseel Alwahidi

Journal of Nursing Ufpe on Line Jnuol Doi 10 5205 01012007

Sebastião Junior Henrique Duarte

Gianluca Catania

RELATED PAPERS

Nudrat Fatima

Diego Aguirre

Agronomía Mesoamericana

Bulmaro Coutiño

Annals of Pharmacotherapy

Atherosclerosis

JOSEPH YEBOAH

International journal of mental health nursing

Michael Olasoji

Diabetes, Metabolic Syndrome and Obesity: Targets and Therapy

simran chhatwal

Journal of human kinetics

Miodrag Spasic

Hafizaitul Husni Liza

Evidence-Based Complementary and Alternative Medicine

Armin Khavandegar

Carcinogenesis

R. Beliveau

Revista chilena de nutrición

Janaína Cover

Materials Chemistry Horizons , Mahsa Ghovvati

Journal of Electronic Materials

Turkish Neurosurgery

Salim Senturk

Gemma Bulos

Life Sciences

effat alemzadeh

The North American Journal of Economics and Finance

António Afonso

E3S web of conferences

Madin Shereuzhev

Journal of Electrostatics

Jeremy Smallwood

Tobias Kröll

Journal of Clinical Medicine

Filipa Pinto-Ribeiro

IABSE Symposium, Guimarães 2019: Towards a Resilient Built Environment Risk and Asset Management

RELATED TOPICS

  •   We're Hiring!
  •   Help Center
  • Find new research papers in:
  • Health Sciences
  • Earth Sciences
  • Cognitive Science
  • Mathematics
  • Computer Science
  • Academia ©2024

Module One Case Study Guidelines and Rubric

case study discussion rubric

USING RUBRICS AND CONTENT ANALYSIS FOR EVALUATING ONLINE DISCUSSION: A CASE STUDY FROM AN ENVIRONMENTAL COURSE

  • Adham R. Ramadan

This paper presents a case study of using course-specific rubrics combined with content analysis, together with instructor and student feedback, to assess learning via online discussion. Student feedback was gathered via Small Group Instructional Diagnosis, and instructor feedback was collected through formal interviews. Content analysis used emergent coding with different assessment criteria for each phase of the online discussion. Student participation was high, with a number of students feeling they learned beyond what was discussed in class. Some students however were overloaded by the large number of postings and repetitiveness during some of the phases of the discussion. The instructor was pleased to find students who were quiet in class being active in the online discussion. However, he found that student contributions demonstrated insufficient reflection and critical thinking. Content analysis showed that students met, on average, 59-82% of the essential assessment criteria in their postings, and that their contributions significantly improved as the online discussion progressed. However, a limited number of postings reflected critical thinking. In using online discussion, the use of assessment criteria is therefore commendable, as it was found that content analysis gave an insight beyond student and instructor perceptions. The insights gleaned from the methodology indicate its usefulness in assessing online discussion activities more objectively, and with respect to specific learning objectives.

McConnell, D. Implementing Computer Supported Cooperative Learning, 2nd Ed., London: Kogan Page, 2002.

Hiltz, S. R. Impacts of college-level courses via Asynchronous Learning Networks: Some Preliminary Results. Journal of Asynchronous Learning Networks 1(2): 1–19, August 1997. Online http://www.sloan-c.org/publications/jaln/v1n2/pdf/v1n2_hiltz.pdf .

Green, L. Online Conferencing: Lessons Learne, 1998. Online http://www.emoderators.com/moderators/lessonse.pdf .

Meyer, K. The Ebb and Flow of Online Discussions: What Bloom Can Tell Us About Our Students’ Conversations. Journal of Asynchronous Learning Networks 9(1): 53–63, March 2005. Online http://www.sloan-c.org/publications/jaln/v9n1/pdf/v9n1_meyer.pdf .

Hammond, M. A Review of Recent Papers on Online Discusson in Teaching and Learning in Higher Education. Journal of Asynchronous Learning Networks 9(3): 9–23, 2005. Online http://www.sloan-c.org/publications/jaln/v9n3/pdf/v9n3_hammond.pdf .

Bali, M. and A. Ellozy. Does WebCT Enhance Learning? Case Studies at AUC. 3rd International E-learning Conference, Egypt, January 2005.

Preece, J., B. Nonnecke and D. Andrews. The top 5 reasons for lurking: Improving community experiences for everyone. Computers in Human Behavior 20(1): 201–223, 2004.

Offir, B., I. Barth, I. Lev and A. Shteinbok. Teacher–student Interactions and Learning Outcomes in a Distance Learning Environment. The Internet and Higher Education 9(2): 65–75, 2003.

Nisbet, D. Measuring the Quantity and Quality of Online Discussion Group Interaction. Journal of eLiteracy 1(2): 122–139, 2004.

COHERE. Briefing on Blended Learning, 2004. Online: http://www.cohere.ca/briefing.html . Wu, D. and S. R. Hiltz. Predicting Learning from Asynchronous Online Discussions. Journal of Asynchronous Learning Networks 8(2): 139–152, April 2004. Online http://www.sloan-c.org/publications/jaln/v8n2/pdf/v8n2_meyer.pdf .

Fredericksen, E., A. Pickett, P. Shea, W. Pelz and K. Swan. Student Satisfaction and Perceived Learning with Online Courses: Principles and Examples from the SUNY Learning Network. Journal of Asynchronous Learning Networks 4(2): 7–41, September 2000. Online http://www.sloanc.org/publications/jaln/v4n2/pdf/v4n2_fredericksen.pdf .

Young. A. and C. Norgard. Assessing the Quality of Online Courses from the Students’ Perspective. The Internet and Higher Education 9(2): 107–115, 2006.

Monroe, B. Fostering Critical Engagement in Online Discussions: The Washington State University Study, 2003. Online: http://www.evergreen.edu/washcenter/Fall2003Newsletter/Pg31-33.pdf .

Hein, T. L. and E. S. Irvine. Assessment of student understanding using on-line discussion groups. Proceedings of Frontiers in Education Conference, Tempe, Arizona, USA, 130–135. (Nov. 4-7,1998). Online: http://ieeexplore.ieee.org/iel4/5943/15885/00736819.pdf?arnumber=736819 .

Hara, N., C. J. Bonk and C. Angeli. Content Analysis of Online Discussion in an Applied Educational Psychology. Instructional Science 28(2): 115–152, 2000.

Meyer, K. Evaluating Online Discussion: Four Different Frames of Analysis. Journal of Asynchronous Learning Networks 8(2): 101–114, April 2004. Online: http://www.sloan-c.org/publications/jaln/v8n2/pdf/v8n2_meyer.pdf .

Roblyer, M. D. and W. R. Winecke. Exploring The Interaction Equation: Validating a Rubric to Assess and Encourage Interaction in Distance Courses. Journal of Asynchronous Learning Networks 8(4): 24–37, December 2004. Online: http://www.sloan-c.org/publications/jaln/v8n4/pdf/v8n4_roblyer.pdf .

CSU. Rubric for Online Instruction, CSU, Chico. (2003) Online: http://www.csuchico.edu/celt/roi/ .

Henri, F. Computer conferencing and content analysis. In A Kaye (Ed.) Collaborative Learning through Computer Conferencing: The Najaden Papers, 117–136. London: Springer-Verlag, 1992.

Lally V. Analysing teaching and learning in networked collaborative learning environments: Issues and work in progress. In V. Lally and D. McConnell (eds.), Networked Collaborative Learning and ICTs in Higher Education: The Edinburgh Papers, 5–26. Sheffield: School of Education University of Sheffield, 2002a.

Fahy, P. J., G. Crawford and M. Ally. Patterns of Interaction in a Computer Conference Transcript. International Review of Research in Open and Distance Learning 2(1): 2001. Online: http://www.icaap.org/iuicode?149.2.1.4 .

Gunawardena, C. N., C. A. Lowe, and T. Anderson. Analysis of a global debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research 17(4): 397–431, 1997.

Parlett, M. and D. Hamilton. Chapter 1.1: Evaluation as illumination: a new approach to the study of innovatory programmes. In D. Hamilton, D. Jenkins, C. King and B. MacDonald (Eds.), Beyond the Numbers Game, 6–22. London: Macmillan Education Ltd., 1977.

White, K. Mid-Course Adjustments: Using Small Group Instructional Diagnoses To Improve Teaching and Learning. In Washington Center's Evaluation Committee (Ed.) Assessment in and of Collaborative Learning, 1995.Online: http://www.evergreen.edu/washcenter/resources/acl/c4.html .

Laurillard, D. What students bring to learning. In Rethinking University Teaching: A Framework for the Effective Use of Educational Technology, 30–47. London: Routledge, 1993.

Pearson Education. The Advantages of Rubrics: Part one in a five-part series, 2005. Online: http://www.teachervision.fen.com/page/4522.html .

Morse, K. Does One Size Fit All? Exploring asynchronous learning in a multicultural environment. Journal of Asynchronous Learning Networks 7(1): 37–56, February 2003. Online: http://www.sloan-c.org/publications/jaln/v7n1/pdf/v7n1_morse.pdf .

As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/ .

This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.

Author(s) hold copyright in their work, and retain publishing rights without restrictions

Scopus Citescore 2022

The DOAJ Seal is awarded to journals that demonstrate best practice in open access publishing

OLC Membership

Join the OLC

OLC Research Center

case study discussion rubric

Information

  • For Readers
  • For Authors
  • For Librarians

More information about the publishing system, Platform and Workflow by OJS/PKP.

ORIGINAL RESEARCH article

Research on an evaluation rubric for promoting user’s continuous usage intention: a case study of serious games for chinese cultural heritage.

Pingting Mao

  • 1 Department of Design and Manufacturing Engineering, Jeonbuk National University, Jeonju-si, Republic of Korea
  • 2 Department of Industrial Design, Jeonbuk National University, Jeonju, Republic of Korea

The sustainable development of serious games dedicated to Chinese cultural heritage faces challenges due to the lack of prolonged user engagement and iterative optimization based on long-term user feedback. This situation not only hinders the sustainable growth of these games but also limits the transmission of Chinese cultural heritage, a problem that demands urgent attention yet remains under-acknowledged. This study synthesizes literature to unearth user needs from three dimensions: motivational use, quality requirements of games, and continuous usage intention. It identifies 14 influential factors, including cognitive satisfaction, immersion satisfaction, and achievement satisfaction. Drawing from the User Experience Rubric for Educational Games-CH (UEREG-CH) evaluation rubric and the Delphi method, these factors are expanded into an evaluation system model comprising six primary indicators, 14 secondary indicators, and 38 tertiary indicators. The analytic hierarchy process (AHP) is utilized for expert scoring to assign weights to each indicator. This is combined with the fuzzy comprehensive evaluation method to arrive at a final user score of Z = 3.4228, indicating that the evaluation rubric is close to ‘good’ and has received positive user feedback. Ultimately, this generates an evaluation rubric tailored to the specific context of serious games for cultural heritage. By integrating qualitative and quantitative methods, this paper confirms the scientific and rational nature of the evaluation rubric. The study aims to establish a user-approved rubric that encourages continuous usage intention, thereby providing effective guidance for game developers and assisting users in selecting appropriate games, while also addressing the theoretical gap in the field of evaluation for serious games related to cultural heritage.

1 Introduction

This research focuses on Serious Games for Cultural Heritage, which are games developed through the integration of cultural heritage and serious gaming. The primary purpose of these games is to protect and disseminate a specific theme of cultural heritage, primarily through digital interactive formats. They often simulate and emulate traditional cultural skills. In this study, they are abbreviated as CH-SGs ( Zhou and Xu, 2017 ). The interest in the cultural heritage domain within serious games arises from the current social context, where protecting and inheriting cultural heritage is a crucial historical mission for the national defense of independence and sovereignty. Many Chinese cultural heritage projects have been lost due to historical reasons and the complexity of their forms. Therefore, there is an urgent need to broaden the channels for disseminating cultural heritage and to find carriers for its living inheritance and innovation ( Teng, 2019 ). CH-SGs, with their unique functional attributes, make cultural heritage knowledge and skills more accessible to users through digital network technologies and entertaining gaming methods, thereby stimulating their autonomous learning ( Qin, 2017 ). Thus, CH-SGs play an essential role in preserving the authenticity of cultural heritage, enriching its presentation forms, lowering the barriers for user engagement, and assuming the crucial responsibility for the transformative development of cultural heritage. Hence, this paper emphasizes the importance of optimizing and improving CH-SGs for significant social relevance ( Cai, 2020 ). Although CH-SGs possess positive social value and have become a new medium for presenting Chinese traditional culture to the world, supported strongly by the state, their practical development is not optimistic. Their development and usage in the Chinese gaming market are relatively scarce ( Chen and Liu, 2019 ). To date, only about twenty games have been developed by renowned companies like Tencent Games and the Palace Museum ( Figure 1 ). These games only offer a “one-time” experience of cultural heritage, sparking only brief interest in users and failing to elicit long-term usage feedback and real, objective evaluation standards to effectively improve game quality ( Li, 2023 ). Likewise, theoretical research on these games is also scarce. In previous literature, the Chinese scholar Xu Haifeng focused on the Shenyang Imperial Palace’s material culture as a theme, emphasizing the game’s functional and aesthetic attributes to enhance user experience ( Xu H. F., 2019 ). Li Haishi, through an analysis of games for intangible cultural heritage, summarized three design transformation paths, emphasizing the design expression of intangible cultural heritage, its visual transformation, and experience design ( Li, 2019 ). Yang Xiao took Shaman culture as a starting point to analyze the factors influencing cognition to behavior in serious games, proving that games play a more effective role in the dissemination of intangible heritage ( Yang, 2020 ). Previous literature has focused on CH-SGs’ cultural themes, game element design, and dissemination impact, but lacks consideration on how to promote continuous usage intention and exploration of user evaluation of such games.

www.frontiersin.org

Figure 1 . Serious games for Chinese cultural heritage.

The current evaluation theories for serious games generally adopt Gardner and Hatch, (1989) and Krathwohl et al. (1956) . These evaluations focus on curriculum standards and the educational attributes of educations ( Shen, 2015 ). However, established serious game evaluations are mostly limited to educational and medical fields, and their applicability in the cultural heritage domain remains unverified. Therefore, the research on a CH-SGs evaluation rubric in this paper aims to fill this gap, promoting the development of both theoretical and empirical research on CH-SGs evaluation. It also aids other types of serious games in exploring suitable evaluation indicators and assessment tools. This study posits that with the guidance of evaluation rubrics, the design process of CH-SGs is akin to having a measuring ruler and a direction, leading to breakthroughs in research on design, development, and application evaluations of CH-SGs ( Wang et al., 2019a ). An evaluation rubric is a scoring tool that includes standards for assessing a task, encompassing various indicators each with its specific weight and assessment criteria. It is a scoring tool that can present indicators in the form of charts or tables. The Chinese scholar Li and Zhu (2017) considers rubrics as structured quantitative evaluation tools, useful for assessing student digital works with strong operability and accuracy, allowing evaluations by teachers, students, and peers ( Li and Zhu, 2017 ). The evaluation rubric in this paper includes various levels of indicators based on theoretical foundations, representing a comprehensive set of evaluation indicators. It revolves around the needs of CH-SGs users, serving as a reference for designers. Establishing a CH-SGs evaluation rubric requires more than a single theoretical construction; it necessitates examination from the user’s perspective ( Xu L. Q., 2019 ). However, it is observed that the existing evaluation rubrics are predominantly used by teachers, designers, and parents, with few specifically formulated for users. The primary evaluators of these games are educators and developers, not the learners or users. This study argues that the users are the true evaluators of serious games, which requires developers to consider game content that matches user needs right from the design stage ( Shen and Zhang, 2014 ). However, accurately grasping user needs is challenging, and past qualitative evaluations based on user experience have overly relied on the evaluators’ quality, often being subjective and lacking scientific rigor ( Jin and Yang, 2017 ). Therefore, this study attempts a multidimensional exploration of user need indicators and a multi-method validation of the evaluation rubric.

In summary, this research focuses on two main questions: how to construct a multi-dimensional CH-SG evaluation rubric that meets user needs, and whether the constructed CH-SG evaluation rubric is accepted by users. This article summarizes 14 user demand factors through literature induction, from the perspectives of usage motivation, game quality, and continuous usage intention; refers to the User Experience Rubric for Educational Games-CH (UEREG-CH) evaluation scale to initially form a CH-SG evaluation system model with 6 primary indicators, 14 secondary indicators, and 32 tertiary indicators; revises and perfects this model in terms of dimensions and semantics through the Delphi expert consultation method, expanding the original 32 tertiary indicators to 38; then conducts Analytic Hierarchy Process (AHP) hierarchical analysis with expert scoring to assign weights to each indicator of the model; subsequently, through user questionnaire surveys, scores are assigned to this model, combined with the weights given by experts to perform fuzzy comprehensive calculations, obtaining the final user evaluation value to determine whether users accept the CH-SG evaluation scale ( Figure 2 ).

www.frontiersin.org

Figure 2 . Research flowchart.

This study constructs the evaluation rubric for Cultural Heritage Serious Games (CH-SGs) from three dimensions: usage motivation, game quality, and continuous usage intention. Initially, one issue with CH-SGs is that they fail to prioritize users’ motivational needs ( Tang, 2022 ). Instead, from design to content, these games often align with the aesthetics of designers and professionals, as evidenced by games like “Tenon and Mortise,” awarded as an excellent app by the Apple App Store in 2014, “Folding Fan” winning the “2018 Zcool Award - Interface Design Category” special award, and “Nishan Shaman” receiving the “ Lina and Haifeng, 2020 China Best Sound Effect Award” among others. While these games are recognized in the industry, they struggle to captivate users, often described as “one-time games” or “beautiful but useless,” indicating that designers are not fully aware of the real motivations of users. In terms of usage motivation literature, Chinese research on game users, especially focusing on the psychological motivations of adolescents using online games, is limited ( Chen, 2023 ). The scholar Zhong Zhijin found that players engaged in online games are influenced by different motivations like personal achievement, enjoying a social life, and immersing in virtual worlds to escape reality, all significantly boosting game duration and stickiness ( Zhong, 2011 ). Research by Ye Na, involving an empirical survey of 546 college students playing online games, showed that the relationship between achievement satisfaction and online game addiction was the strongest ( Ye et al., 2009 ). This study references Zhang Wenmin’s research on Multiplayer online battle arena (MOBA) mobile game users’ continuous usage intentions, which posits that social, competitive, immersive, and cognitive motivations significantly impact users’ continued usage ( Zhang, 2019 ). Furthermore, the scholar Holbrook noted that the product provided by online games is virtual, offering psychological and emotional satisfaction through emotional interaction with others, leading to leisure enjoyment ( Holbrook and Hirschman, 1982 ). In her empirical study on the relationship between customer perceived value, satisfaction, and loyalty in online games, Hu (2013) also found that leisure enjoyment positively impacts customer retention. Therefore, based on the above discussions, this study incorporates cognitive satisfaction, immersive satisfaction, achievement satisfaction, social satisfaction, and leisure satisfaction into the user motivation indicators for measurement, defined as follows:

1. Cognitive satisfaction: Kahn believes this involves helping users develop specific intellectual abilities through games.

2. Immersive satisfaction: Yee defines this as the behavior or psychological state where users immerse themselves in the virtual world of the game, escaping reality.

3. Achievement satisfaction: This refers to the user’s strong desire to achieve victory in the game.

4. Social satisfaction: Yee sees this as the user’s expectation to achieve social activities on the gaming platform.

5. Leisure satisfaction: Holbrook suggests that through emotional interaction in online games, customers experience a sense of well-being in their consciousness, meeting their psychological and emotional needs and expectations.

Secondly, as products of the internet era, CH-SGs require support from various sectors of the industry chain, from pre-launch advertising by game operators, provision of downloads by distributors, and stable service from service providers to continuous updates and optimization by developers ( Jiang, 2011 ). Users may choose to continue or abandon a game due to various factors such as server quality and system stability. A prominent issue in the current design of CH-SGs is the simplicity of their mechanism design, lacking in gameplay and intrigue, and their weak playability. Additionally, the integration of educational elements with gameplay is insufficient ( Lei and Feng, 2020 ). For instance, games like “Tenon and Mortise” and “Folding Fan” by Tag Design team are termed as “serious games,” but they focus more on conveying the artistry of cultural heritage, with more emphasis on aesthetics than on interactive gaming, failing to convey information through game mechanics or narratives, which points to a deficiency in the game quality of CH-SGs ( Zhang S. Y., 2023 ). Among the literature related to game quality, many scholars view games as a new type of information system and have incorporated the revised model of information system success into studies on game user engagement, confirming that system quality, information quality, and service quality influence user participation ( Wang, 2010 ). This study refers to the scholar Zhang (2016) , who analyzed mobile game user churn and attributed game factors to system quality, service quality, game design, and brand image, demonstrating significant impacts on the perceived usefulness and ease of use by game users ( Fang, 2023 ). Furthermore, CH-SGs differ from traditional online games in that they are primarily designed to address real-world social and industrial problems. They are not solely for entertainment but are meant to achieve at least one additional goal, namely, the learning objective, without compromising the user’s gaming experience ( Peng and Liu, 2020 ). Many studies have explored different quality standards for serious games, including effectiveness and attractiveness, but have not fully considered the dual mission of serious games, i.e., achieving both functional and gaming aspects. High-quality serious games must balance functionality and gaming; they must systematically support users in achieving learning goals (functional part) and must engage and maintain entertainment objectives (gaming part). These two aspects should be perfectly matched and integrated, rather than being addressed in isolation ( Caserman et al., 2020 ). Therefore, this paper also considers learning and entertainment objectives as factors in game quality, defined as follows:

1. System quality: This refers to the stability and usability of the game system, essential for ensuring a normal gaming experience for users.

2. Service quality: This assesses the quality of services provided by companies to users, ensuring a comfortable and enjoyable gaming experience.

3. Game design: This involves the rational organization and arrangement of various elements of a game product, creation through game engines, and the final formation of the game product.

4. Brand image: The perceived influence of the game brand by users, including the brand image of the game itself and the company’s reputation.

5. Learning objectives: The learning objectives of CH-SGs should be closely related to their application fields. The learning or training effect should remain the focus throughout the game process, always supporting users in achieving this purpose, with game elements not interfering with the learning or training process.

6. Entertainment objectives: High-quality CH-SGs should be engaging and enjoyable, focusing on user involvement, positive gameplay experiences, fluidity, and feeling of control, as well as supporting social interaction to ensure users find enjoyment in the game.

Additionally, this study emphasizes the promotion of continued usage of CH-SGs. Parthasarathy once noted in the context of information systems that retaining an existing customer costs five times less than acquiring a new one ( Yu and Cheng, 2012 ). Long-term user feedback helps understand the ever-changing needs of users, improving and innovating game products. Continued usage intention is defined as the subjective desire of users to continue using an information system after initial use ( Niu, 2020 ). In the field of continued usage intention, the technology acceptance model (TAM) has been widely applied and has become one of the most mature theories in this area ( Sun, 2023 ). As online games, fundamentally, belong to the category of information systems ( Li, 2011 ), many scholars have used the TAM model in their studies of online game users’ continuous usage ( Min, 2018 ). The TAM model predicts the degree to which a person accepts, uses, or rejects information technology, effectively explaining the behavior of users in adopting new technologies and its influencing factors ( Sun and Guo, 2023 ), and is the most concise and universal model for studying the initial intention and continuous intention to adopt technology ( Yao, 2023 ). Davis (1989) first proposed the TAM model while studying users’ acceptance of computers. He suggested that perceived usefulness is an extent indicator, reflecting the individual’s perception of how an information system enhances their work efficiency. Higher perceived usefulness increases the likelihood of taking actions; perceived ease of use reflects the individual’s perceived difficulty in using the system, with higher ease of use leading to more likely user behavior ( Jia, 2019 ). Later in 2000, Davis and Venkatesh revised the TAM to include subjective norms based on numerous studies ( Dong, 2017 ). Scholars like Sun Shaojun integrated the TAM and Stimulus-Organism-Response (SOR) models to establish a continuous usage behavior model for game live streaming ( Sun and Zhang, 2017 ); Wu Xuanying integrated the TAM model and flow theory to construct a stickiness influence model for mobile game users ( Wu, 2019 ); Zhang Di combined the SOR, TAM, information success model, and immersion theory to build a model for mobile game user churn factors ( Zhang, 2016 ). These studies confirmed the positive impact of perceived usefulness, ease of use, and subjective norms on continuous usage. Therefore, this study incorporates these three variables into the continuous usage indicators, defined as follows:

1. Perceived usefulness: The degree to which a user perceives that using a particular information system will help them perform tasks better or gain other benefits. When users perceive that using such games meets their expected benefits, they will feel satisfied and continue using the games.

2. Perceived ease of use: The degree to which a user perceives the ease or difficulty of using a particular information system. If users encounter difficulties when experiencing a game, they may feel dissatisfied and abandon it, whereas if they find the game easy to operate, they will feel satisfied and continue using it.

3. Subjective norms: The degree to which a user’s behavior is influenced by group dynamics, such as conformity, suggestion, and obedience. Therefore, an individual’s behavior is often influenced by the group. If users perceive that no one in their social circle is using the game, they may feel that the game is futile and abandon it due to a lack of positive gaming experience.

2 Methodology

2.1 constructing the ch-sg evaluation system model.

This study posits that only by establishing an evaluation rubric fitting the CH-SG context can effective assessments be conducted in all aspects, referencing the UEREG-CH evaluation scale with high similarity prior to construction ( Table 1 ). This scale was developed by the team at the Educational Game Center of Nanjing Normal University. It is guided theoretically by Hartmann’s improved model of user experience influencing factors, reviews the current state of research on factors influencing user experience, and analyzes the characteristics and elements of serious game user experiences. The rubric identifies key terms in user experience evaluation through user testing, and then refines the primary and secondary indicators of the index system through two rounds of expert consultation, thus ultimately determining this evaluation rubric ( Shen, 2015 ).

www.frontiersin.org

Table 1 . The UEREG-CH evaluation rubric.

This paper, based on literature research, identifies 14 influencing factors: ‘Intellectual satisfaction, Immersion satisfaction, Achievement satisfaction, Social satisfaction, Pleasure satisfaction, System quality, Service quality, Game design, Brand image, Learning objectives, Entertainment objectives, Perceived usefulness, Perceived ease of use, and Subjective norms’. These factors are used as secondary indicators in the CH-SG evaluation system model. Referring to the dimensions in the UEREG-CH evaluation rubrics, these 14 secondary indicators are categorized into 6 primary indicators and further expanded into 32 tertiary indicators, thereby preliminarily forming the CH-SG evaluation system model ( Liu et al., 2021 ) ( Table 2 ).

www.frontiersin.org

Table 2 . Initial establishment of the CH-SG evaluation system model.

2.2 The Delphi method

The Delphi method is a group decision-making behavior that involves a series of concentrated expert surveys. Based on the collective professional knowledge and experience of experts, supplemented by controlled feedback of opinions, the process aims to achieve the greatest consensus among a group of experts ( Dai, 2023 ). The advantage of the Delphi method is that it avoids direct confrontation between experts and structures the discussion process of the expert group, making them more effective when discussing complex issues ( Liu et al., 2021 ). It is not only used in the field of prediction but also widely used in the establishment of various evaluation systems and the determination of specific indicators ( Liu et al., 2011 ). The 10 experts who filled out the questionnaire on game studies were strictly selected and all had at least 2–3 years of research experience in serious games, educational games, and online games, with at least 2 years of game testing experience and at least one experience in educational game evaluation ( Liu et al., 2023 ).

Ten experts were consulted over two rounds regarding the initially established CH-SGs evaluation system model, using both electronic mail and paper questionnaires to collect expert opinions and to add, delete, or modify indicators. The specific process included: clarifying evaluation objectives, selecting experts, issuing questions, experts’ assessment of questions, and revising based on expert feedback ( Liu et al., 2011 ). The first round of questionnaires involved experts’ familiarity with each level of indicators, suggestions for additional or deletable indicators, and other questions or recommendations. Based on expert suggestions, similar indicators in the CH-SG evaluation system model were merged, unreasonable ones were deleted or broken down, and missing indicators were added ( Dong, 2013 ). In the first round of questionnaire consultation, the primary indicator “Identity dimension” was not described accurately and was loosely connected logically with the secondary indicators “Brand image” and “Subjective norms,” hence it was changed to “Social dimension.” The secondary indicator “Game design” was too broad and encompassed too much, so it was changed to “Interface design” based on the tertiary indicators. Complex descriptions in tertiary indicators were further refined, such as “D1 Game interface operation setting is reasonable, with the characteristics of the cultural heritage theme,” which was split into “D1 Interface design features cultural heritage theme” and “D2 Interface menu layout is reasonable and orderly” for easier understanding and scoring by experts or users. “D4 Immersed in the game’s artistic roles, creating one’s own story background” was simplified to “D5 Immersion in creating one’s story with artistic roles,” etc. Other less comprehensive descriptions were expanded, thus increasing the original 32 tertiary indicators to 38. Overall, the experts acknowledged the feasibility of the indicators, with minimal modifications made ( Table 3 ).

www.frontiersin.org

Table 3 . Revised CH-SG evaluation system model.

The revised CH-SG evaluation system model consists of 6 primary indicators, 14 secondary indicators, and 38 tertiary indicators. In the second round of questionnaire surveys, the opinions of the experts converged, with the agreement rate for each level of indicators reaching 100%. Therefore, no further modifications were needed, and the CH-SG evaluation system model was finally determined to be used as the AHP for the next phase of research.

2.3 AHP hierarchical analysis

The AHP is a decision-making method for complex and vague issues, particularly suitable for issues difficult to be fully quantified. It is a flexible, convenient, and practical multi-criteria decision-making method ( Zhang et al., 2014 ). The principle of AHP is to organize and hierarchize problems, constructing a structured model with various levels. Complex issues are decomposed into elements, which are then grouped according to their characteristics and certain rules into several levels. Elements within each level have approximately the same status. An orderly, step-by-step hierarchy is established based on the affiliation relationships between levels, where elements of a higher level dominate related elements of the next lower level ( Dong, 2013 ). In this step-by-step hierarchical model, decision-makers score the importance of each level quantitatively based on their judgment of certain objective facts. This is done by determining the relative importance of elements within each level through pairwise comparison and quantitatively expressing this importance, thereby establishing judgment matrices. Mathematical models are then used to calculate the relative importance weights of each indicator within the judgment matrices of each level. Finally, the relative importance weights of all indicators concerning the evaluation objective are calculated through corresponding operations ( Xu, 2009 ). In the CH-SG hierarchical model of this paper, the relationship between primary, secondary, and tertiary indicators is one of containment and being contained, also known as the upper and lower level relationship in the decision-making field. In the hierarchical model, “CH-SG Evaluation System Model” A is the goal layer, B1-B6 are primary indicators, C1-C14 secondary indicators are criterion layers, and D1-D38 tertiary indicators are the scheme layer. Experts compare and score the indicators in the evaluation system ( Table 4 ), generate judgment matrices, and calculate the weight values of each indicator.

www.frontiersin.org

Table 4 . Expert questionnaire scoring 1–9 scale method.

Steps for calculating weights using the AH involve the hypothetical judgment matrix A:

The calculation steps for the root method are as follows:

(1) Calculate the product of the elements in each row of the judgment matrix A:

(2) Calculate the nth root of mi;

(3) Normalize the vector W ∗ = w 1 ∗ w 2 ∗ ⋯ w n ∗ T

The weights obtained are the desired weight vector, w1, w2,..., wn, which corresponds to the weight value of each element.

Matrix consistency test method:

(4) Calculate the consistency index

The calculation formula for the largest eigenvalue is: λ max

(5) Look up the corresponding Average Random Consistency Index RI.

www.frontiersin.org

(6) Average Consistency Ratio (CR).

When CR is less than 0.1, the consistency of the judgment matrix A is considered acceptable; otherwise, some adjustments to the judgment matrix are required ( Zhu et al., 2020 ) (Refer to the appendix for the detailed calculation process).

Based on the scoring from 10 experts, a judgment matrix is constructed, and the results of the weight calculation using the AHP are as follows:

Table 5 shows that the CR value is <0.1, passing the consistency test. The smaller the CR value, the better the consistency of the judgment matrix, indicating the rationality of this weight determination method. There is no need to modify the judgment matrix. The derived weight set can reflect the importance of each indicator, and the weight distribution is reasonable ( Li and Guo, 2004 ).

www.frontiersin.org

Table 5 . Results of the consistency test.

Through the above research, the weight coefficients of each level of indicators for the CH-SGs evaluation were determined ( Table 6 ), forming a complete evaluation system. Subsequently, the fuzzy comprehensive evaluation method will be used by users to assess the overall quality of this system ( Zhang X. W., 2023 ).

www.frontiersin.org

Table 6 . Weight values of each indicator in the CH-SG evaluation system model.

2.4 Fuzzy comprehensive calculation

Fuzzy comprehensive calculation is based on fuzzy mathematics and applies the principle of fuzzy relation synthesis to quantify factors with unclear boundaries and difficult quantification, conducting a comprehensive evaluation of the belonging level of an entity under evaluation from multiple factors ( Ge and Sun, 2014 ). The basic procedure is to first determine the factor domain of the evaluation object, i.e., the number of indicators, and second to determine the evaluation level domain, typically taking integers, and for the convenience of expert judgment, the levels are divided as finely as possible ( Li and Deng, 2019 ). This study adopts an evaluation set of levels: poor, below average, average, good, and excellent. Finally, the weights of each indicator obtained from the AHP are combined to calculate the final fuzzy evaluation result ( Wang, 2017 ). The fuzzy comprehensive calculation is as follows:

1 First, determine the factor domain of the evaluation object (i.e., the fuzzy factor set): let there be p evaluation indicators, U = {u1, u2,..., up}.

2 Determine the weight vector of evaluation factors and the evaluation level domain (i.e., the evaluation set): in fuzzy comprehensive evaluation, determine the weight vector of evaluation factors: W = {a1, a2,..., ap}. The relative importance order of evaluation indicators is usually determined using the AHP or other methods to establish the weight coefficients, and normalize them before synthesis. Let the evaluation level domain (i.e., the evaluation set) be V = {v1, v2,..., vm}, with each level corresponding to a fuzzy subset, i.e., the level set.

3 Establish a fuzzy relation matrix (i.e., the membership degree matrix): after constructing the level fuzzy subsets, quantify the evaluated entity from each factor ui (i = 1, 2,..., p), i.e., determine the membership degree of the evaluated entity to the level fuzzy subsets (R|ui). Through survey questionnaires, obtain the scoring of each person for each level of the indicator, with the proportion of the number of people scoring each evaluation level to the total number of experts as the membership degree, thereby establishing the single factor fuzzy comprehensive evaluation matrix.

The membership degree matrix for a single element (R|ui) = (ri1, ri2,..., rim), where for each level of a single element, the membership degree is: r i j = c i j c .

Cij represents the number of people who chose level vj for indicator i, and c represents the total number of experts participating in the evaluation.

Calculate the membership degree for p evaluation indicators to obtain the overall fuzzy relation matrix, as follows:

In this element, the element rij in the ith row and jth column represents the membership degree of a certain evaluated entity ui in terms of the factor to the level vj fuzzy subset.

4 Synthesize the fuzzy comprehensive evaluation result vector: combine W with the overall fuzzy relation matrix R of each evaluated entity to obtain the fuzzy comprehensive evaluation vector B for each evaluated entity, i.e.:

B=W*R = (a1,a2,…,ap)* r 11 r 12 … r 1 m r 21 r 22 … r 2 m … … … … r p 1 r p 2 … r pm =(b1,b2,…,bm).

bi represents the degree of membership of the evaluated entity to the level vj fuzzy subset from an overall perspective.

5 Analyze the fuzzy comprehensive evaluation result vector: the most commonly used method in practice is the principle of the maximum degree of membership. However, in some cases, its use can be forced, leading to the loss of a lot of information, and even producing unreasonable evaluation results. This paper uses the method of weighted average to determine the membership level, allowing for the ranking of multiple evaluated entities according to their level positions (refer to the appendix for the detailed calculation process).

To verify the scientific and practical nature of the CH-SG evaluation system model discussed earlier, this paper utilizes the weights of indicators determined by the AHP and applies fuzzy comprehensive calculation to evaluate the comprehensive score of this system. For this purpose, a questionnaire targeted at users of CH-SGs is designed (refer to the appendix for details). Evaluators simply need to select the corresponding levels for each indicator on the scale based on their judgment ( Zhang, 2015 ). A total of 400 questionnaires were distributed, with 392 valid responses collected. To ensure the reliability and validity of the questionnaire, the sample data was analyzed using the Cronbach’s alpha coefficient in SPSS, yielding an overall Cronbach’s alpha of α = 0.962, indicating good internal consistency. Exploratory factor analysis was also conducted, with a Kaiser-Meyer-Olkin (KMO) value of 0.978 and a significant level of Bartlett’s test of sphericity at 0.000, indicating significant correlations between variables and passing the test of reliability and validity. This paper analyzes the collected questionnaires, calculating the frequency of each level—poor, below average, average, good, and excellent—and the corresponding values of each indicator for fuzzy comprehensive calculation. Through these calculations, the fuzzy evaluation values of each level of indicators in the CH-SGs evaluation system are derived, providing reference for further analysis of the evaluation rubric.

The fuzzy comprehensive calculation resulted in the evaluation values for each level of indicators in the CH-SG evaluation rubric ( Table 7 ). The overall primary level fuzzy comprehensive evaluation value was 3.4228, indicating an overall level between average and good. This suggests that the evaluation rubric is recognized by users, and the descriptions of evaluations in these games closely align with user experiences ( Liang, 2019 ). According to the data, the primary indicators are ranked by their importance as follows: B4 Gamification at 3.5371 > B5 Demand at 3.4546 > B6 Social dimension at 3.4473 > B3 Educational value at 3.4307 > B1 Artistic dimension at 3.4223 > B2 Usability at 3.2355. The B2 Usability at 3.2355 is notably lower than the overall evaluation value of 3.4228, indicating that this indicator is not ideal and suggests a mismatch between expert and user standards in evaluating the game.

www.frontiersin.org

Table 7 . Fuzzy evaluation values of each indicator in the CH-SG Evaluation Rubric.

The primary indicator B4 Gamification at 3.5371 is higher than the overall comprehensive evaluation value, indicating that enhancing gameplay positively impacts the evaluation of the game and that users are most concerned with the gameplay aspect of CH-SGs. In the secondary indicators, C8 Perceived ease of use at 3.6105 and C10 Pleasure satisfaction at 3.5510 are higher than the gamification indicators, enhancing the gamification. In the tertiary indicators, D21 Easy download, installation, payment for the game at 3.6301, D22 Easy learning and operation of the game at 3.5714, D23 Establishing multi-modal experiences such as graphics, sound effects, and tactile at 3.7883, and D29 Participation in offline experiences and purchasing cultural peripheral products at 3.6633 are all higher than the gamification, having a positive impact. Designers should consider how to make the game user-friendly, aligning with the characteristics of CH-SGs as light-function games, requiring minimal time and effort from users, facilitating knowledge acquisition, character cultivation, enjoyment, and mental pleasure, and providing different experiences in sound effects, tactile modes, and offline activities, all of which will effectively increase users’ willingness to continue using the game ( Gong and Liang, 2018 ).

The primary indicator B5 Needs at 3.4546 is higher than the overall evaluation value, positively impacting it. In its secondary indicators, C12 Achievement satisfaction at 3.4719 is slightly higher than the demand dimension. In the tertiary indicators, D31 Making many friends through cooperation and communication with other users at 3.4745, D32 Long-term contact with other users and feeling care among friends at 3.8954, and D33 The desire to win in challenges and competitions at 3.5434 are all higher than the demand dimension, positively impacting it. Designers should focus on expanding users’ social pathways and convenience, fully satisfying users’ desires for social interaction and emotional communication, and increasing the game’s competitiveness and positive feedback to stimulate users’ sense of achievement, thereby promoting their willingness to continue using the game ( Chang, 2015 ).

The primary indicator B6 Social dimension at 3.4473 is higher than the overall evaluation value, positively impacting it. In its secondary indicators, C13 Brand image at 3.5485 is higher than the social dimension indicator. In the tertiary indicators, D36 Production companies with high recognition or liked experts and stars at 3.7474 and D35 High recognition or liked cultural IP themes at 3.4490 are both higher than the primary indicator. This indicates that production companies with high recognition are a guarantee of high-quality games for users. Choosing well-known experts and stars for advertising endorsements can effectively attract users’ attention. Alternatively, choosing popular, well-received IPs for game development can arouse users’ curiosity and encourage continued use ( Wang, 2022 ).

The primary indicator B3 Educational value at 3.4307 is slightly higher than the overall evaluation value, also having a promotive effect. In its secondary indicators, C5 Learning objectives at 3.5879 and C6 Intellectual satisfaction at 3.6956, are higher than the primary indicator. In the tertiary indicators, D12 Continuous and systematic learning content at 3.7908, D15 Accurate and reliable cultural heritage knowledge in various forms at 3.8316, D16 Level design conforming to learning rules and user cognitive level at 3.4413, D17 Enhancing the ability to complete tasks and cooperate intellectually with others at 3.6939, and D18 Stimulating curiosity in learning traditional culture at 3.6990, all these indicators are higher than the primary indicator. The educational value is where serious games differ from other entertainment games in value, thus gaining more attention from users ( Li and Sun, 2019 ). Designers, while developing the educational goals of the game, must first ensure that the integrated cultural heritage knowledge is accurate and diverse, maintaining users’ freshness toward knowledge. Secondly, they must consider whether the designed cultural knowledge and skills are systematic, enabling users to learn progressively rather than a one-time experience that cannot be sustained. They must also consider whether the knowledge broken down into stages of task levels conforms to the users’ cognitive level, ensuring that users can overcome the task with some effort or seeking help, without feeling a sense of failure and giving up. Finally, users should gain from the use and learning of the game, either through increased positive rewards or by setting stage-wise tasks and evaluation rewards, ensuring that users have gains at every stage after completing tasks or cooperating with other users, and developing new cognition and curiosity about cultural heritage. These aspects can effectively stimulate users to continue using the game ( Huang, 2023 ).

The primary indicator B1 Artistic dimension at 3.4223 is close to the overall evaluation value. In its secondary indicators, C2 Immersion satisfaction at 3.6786 is higher than the artistic dimension, positively impacting it. In the tertiary indicators, D4 High clarity, exquisite realism, and interesting visual scenes at 3.6709, D5 Immersion in creating one’s own artistic characters and stories at 3.6378, and D6 Helps to forget problems and stresses of real life at 3.8010, all these indicators are higher than the primary indicator value. This suggests that users hope to immerse themselves in a virtual world through the game, temporarily escaping life’s troubles for mental rest. To achieve such immersion ( Wang et al., 2019b ), designers need to enhance the visual effects of game scenes and endow characters with artistry, strongly attracting users’ attention, and giving them more control to create their own characters and stories ( Zhang, 2018 ). Stimulating these aspects can enhance users’ willingness to continue using such games.

The primary indicator B2 Usability at 3.2355 is below the overall evaluation value, indicating that this indicator is not ideal and that there is a mismatch between experts’ and users’ standards in evaluating the game. In its secondary indicators, C4 Service quality at 3.5000 is higher than both the primary indicator and the overall evaluation value, suggesting that users are more concerned about the service attitude and quality of the game production company. In its tertiary indicators, D10 Conducting promotional activities and fulfilling promises to users at 3.4745 and D11 Customer service staff being competent and promptly resolving user issues at 3.5510, these indicators indicate that if game production companies deceive users during use or fail to fulfill various promotional promises, or if customer service staff have a poor attitude, lack professional competence, or do not promptly resolve users’ issues in the game, it will lead to user dissatisfaction and the desire to quit the game.

4 Discussion

4.1 comparative analysis of ch-sgs and uereg-ch setting principles.

This section compares and analyzes the CH-SGs evaluation model and UEREG-CH based on three principles of rubric setting: purposefulness, systematicity, and measurability. Purposefulness is the fundamental basis for establishing evaluation rubrics and directly affects the conduct of evaluation activities. Systematicity requires that the evaluation rubric should comprehensively reflect the requirements of the evaluation objective, without missing any significantly important indicators. Additionally, indicators should be mutually independent, without causal or overlapping relationships, clearly hierarchical, and collectively form an organic whole. Measurability demands that specific indicators should be defined in operationalizable language, allowing for conclusions to be drawn through actual observation or measurement. This principle concretizes and quantifies abstract evaluation objectives, and is one of the basic conditions for the practical application of evaluation rubrics and the generation of reliability ( Shen, 2015 ).

Purposefulness: The UEREG-CH primarily reflects the sensory experiences and real needs of serious game users, including students and teachers. The purpose of the evaluation is to select educational games that meet the autonomous learning needs of learners and adapt to different learning styles. The CH-SGs in this study primarily focuses on user needs and experiences to promote continued use. It provides scientific and reasonable guidance and evaluation standards for designers and developers of these games and assists users and learners in selecting suitable CH-SGs.

Systematicity: The UEREG-CH, centered on user experience as the evaluation objective, sets up 5 primary indicators, 22 secondary indicators, and corresponding descriptions and weights, with a tightly interconnected hierarchical relationship. For example, under “Gamification,” there are secondary indicators like “Appropriate challenge” and “Reasonable incentives.” However, a shortcoming is the similarity or lack of distinction between some secondary indicators, such as “Organized interface menu” and “Clear interface design”; and “Easy to remember” and “Easy to learn.” CH-SGs in this study sets up 6 primary indicators, 14 secondary indicators, and 38 tertiary indicators, assigning weights to each. Focusing on user experience, it emphasizes promoting continued use and includes content related to user motivation and game quality. The considered range of indicators is broader, deeper, and more detailed, with each level of indicators being interrelated yet distinct and easy to differentiate.

Measurability: Both the UEREG-CH and CH-SGs achieve clarity and conciseness in the design of indicator descriptions. In designing the scale method, descriptive terms like “excellent,” “good,” “average,” “poor,” and “below average” are used. Evaluators score according to the degree to which the last level of indicators meets the description.

4.2 Comparative analysis of CH-SGs and UEREG-CH indicator dimensions

4.2.1 gamification.

The UEREG-CH breaks down “Gamification” into five secondary indicators: challenge, reward, content, plot, and rules, which lacks comprehensive consideration. This study dissects the “Gamification” into “Perceived ease of use,” “Entertainment objective,” and “Pleasure satisfaction,” along with nine tertiary indicators from D21 to D29 ( Table 8 ). It posits that the prerequisite for user engagement in a game is its perceived ease of use, as simple operability leads users to further explore the game’s entertainment aspects. The “Entertainment objective” involves users empathizing with characters, controlling the game’s direction, achieving self-realization, and immersing in the experience, leading to the satisfaction of pleasure and making the game a part of their life. This study views gamification as a distinguishing feature of CH-SGs compared to other educational games, necessitating a multi-faceted consideration of users’ actual operational needs.

www.frontiersin.org

Table 8 . Comparison of indicator dimensions.

4.2.2 Demand

The UEREG-CH decomposes “Demand” into two aspects: “Game learning content meets or exceeds learner needs” and “Game functionality meets or exceeds learner needs.” This study finds this approach overly simplistic and lacking a clear distinction from “Educational” aspects. CH-SGs categorizes the “Demand dimension” into “Social satisfaction” and “Achievement satisfaction,” with five tertiary indicators from D30 to D34 ( Table 8 ). It believes that users’ intrinsic demands stem from two sources: achievement satisfaction through winning in the game mechanics, and social satisfaction from connections made with other users. These needs are interdependent, as victory in the game leads to a desire for social display and sharing, while collaboration and communication with others facilitate winning.

4.2.3 Social aspect

The UEREG-CH does not set this indicator ( Table 8 ). However, this study argues that games, as products, have strong social attributes, encompassing both users’ social attributes and the game’s societal aspects. CH-SG users often belong to culturally and economically higher echelons, greatly influenced by social norms, and their actions are affected by others in society. If close friends or experts recommend such games, users are likely to try them. The “Brand image” represents the game product’s social attribute. If the game’s production company is well-known, endorsed by celebrities or experts, or developed from a well-known IP, it tends to be favored and sought after by users.

4.2.4 Educational aspect

In the UEREG-CH, “Educational value” is divided into five secondary indicators: “Clear and timely knowledge feedback,” “Levels compliant with learning rules,” “Reliable content with flexible and diverse forms,” “Clear learning objectives,” and “Balanced skills and challenges.” This categorization appears somewhat generic and necessitates further detailed expansion. CH-SGs decompose the “Educational value” into three secondary indicators: “Learning objectives,” “Intellectual satisfaction,” and “Perceived usefulness,” along with nine tertiary indicators from D12 to D20 ( Table 8 ). “Learning objectives” play a crucial role in the quality aspect of CH-SGs games and serve as a distinctive feature differentiating them from other games. Users experience intellectual satisfaction only after completing tasks set by the “Learning objectives.” This sense of intellectual satisfaction, being one of the primary motives for initial game usage, leads to the perception of usefulness, thereby encouraging users to continue using the game. Thus, the relationship among these three indicators is interlinked, each supporting and leading into the next, collectively embodying the educational aspect from multiple perspectives.

4.2.5 Artistic aspect

The UEREG-CH deconstructs “Perceived aesthetics” into interface design, menu, elements, production, etc., which this study believes lacks an overall requirement for the game’s artistry and attention to users’ perception of art. CH-SGs breaks down “Artistic aspect” into two secondary indicators: “Interface design” and “Immersive satisfaction,” along with six tertiary indicators from D1 to D6 ( Table 8 ). This study suggests that the artistic indicators should reflect cultural heritage themes and emphasize creativity more than other educational games. Only when users perceive a unique artistic atmosphere in the interface can they immerse themselves in their game characters and forget the stresses of life.

4.2.6 Usability

The UEREG-CH decomposes “Usability” into indicators such as ease of memorization, operation, learning, absence of faults, and timely feedback. However, these indicators are subjective and semantically vague, overlapping with others. This study breaks down “Usability” into “System quality” and “Service quality,” with five tertiary indicators from D7 to D11 ( Table 8 ). A game’s usability is first judged by system quality; a game is considered usable only when the system is stable, frequently optimized, and has good guidance and synchronous feedback. Additionally, customer service’s timely problem resolution and fulfillment of promises to users are essential; otherwise, users may develop negative perceptions of the game’s usability and abandon it.

4.3 Comparative analysis of indicator weights in CH-SGs and UEREG-CH

4.3.1 weight structure.

Both the UEREG-CH and CH-SGs have tree-like indicator systems, but the former does not highlight the characteristics of various serious games as effectively as the latter, which is tailored for CH-SGs. Regarding the method of weight determination, the UEREG-CH uses group AHP cluster analysis, while the CH-SG combines the Delphi method, AHP, and fuzzy comprehensive calculation, blending qualitative and quantitative research ( Table 9 ).

www.frontiersin.org

Table 9 . Comparison of indicator weights.

4.3.2 Important indicator weights

In the UEREG-CH, “Usability” ranks first, followed by “Educational value,” “Demand,” “Gamification,” and “Perceived aesthetics.” This prioritization shows that serious games in China focus more on basic user needs, emphasizing usability and educational aspects but are weaker in gamification and artistic aspects, lacking a balance between learning and gaming. In CH-SGs, “Gamification” ranks first, followed by “Demand,” “Social aspect,” “Educational value,” “Artistic dimension,” and “Usability.” This suggests that CH-SGs users prioritize leisure and entertainment in games, personal needs fulfillment, and social interaction. However, the educational aspect is less emphasized, possibly because CH-SGs, while educational, are perceived as weaker in function compared to pure educational games. This perception might stem from the games’ design issues that fail to emphasize learning objectives through game mechanics, leading to the impression that their educational function is weak. The lesser importance of the artistic aspect could be due to the convergence of artistic styles in current CH-SGs, predominantly adopting traditional Chinese styles and lacking in diversity and innovation. Usability ranks last because system quality generally meets users’ needs in current CH-SGs, and service quality has a minor impact on user experience, thus receiving less attention from users ( Table 9 ).

5 Conclusion and limitations

This study confirms that an effective and comprehensive evaluation rubric for CH-SGs can be constructed by focusing on usage motivation, game quality, and continuous usage. Previous research on serious games has focused on the educational aspect and the improvement of learners’ outcomes. However, empirical studies primarily aimed at collecting evidence of learning effectiveness have design flaws, leading to a shift in the research trend beyond just educational aspects. This study observes that the educational function of CH-SGs in China is relatively weak and needs to be integrated into the game mechanics to achieve learning objectives. Compared to other serious games, CH-SGs have a distinct cultural and artistic attribute. Therefore, evaluation rubrics developed for other serious games are not suitable for CH-SGs. To reflect the characteristics of CH-SGs and the needs of their user group, this study distinguishes them from other serious games across six dimensions: “Artistic,” “Usability,” “Educational value,” “Gamification,” “Demand,” and “Social aspect.” The secondary and tertiary indicators emphasize cultural heritage elements, such as breaking down the “Educational value” dimension into technical indicators like “Learning objectives” and non-technical indicators like “Intellectual satisfaction” and “Perceived usefulness,” with tertiary indicators reflecting elements of cultural heritage, such as “Accurate and reliable cultural heritage knowledge in various forms,” “Stimulating curiosity in learning traditional culture,” and “Enhancing the level of traditional cultural knowledge and skills.” Empirical research has revealed a departure from the traditional approach in serious game studies, which often emphasizes a balance between educational and gameplay aspects. It was found that users of CH-SGs tend to favor “Gamification” more, with “Educational” factors ranking only fourth in terms of importance. Therefore, it is recommended that designers of CH-SGs tailor the balance between gaming and learning aspects according to the specific theme and type of each game, rather than uniformly pursuing a balance between the two.

This study also validates that the CH-SGs evaluation rubric is recognized by users, not just experts. Involving users in the evaluation process is crucial, as they have the right to assess the games. Therefore, the evaluation rubric is developed around users’ diverse needs, formed through expert consultations and the AHP method, and further validated by user surveys and fuzzy comprehensive calculations. This approach ensures significant user participation. The study also believes that a game’s true appreciation by users depends on their continued use and the lasting, long-term impact it has on them. Thus, the rubric differs from previous studies by including long-term indicators, focusing on users’ experiences, attitudes, enthusiasm, persistence, and other non-technical aspects ( Yu, 2016 ). Starting from the theory of continued usage intention, the study sets “Perceived usefulness,” “Perceived ease of use,” and “Subjective norms” as secondary indicators, and “Continuous and systematic learning content,” “Long-term contact with other users and feeling care among friends,” etc., as tertiary indicators to assess the long-term impact of CH-SGs on users. True feedback for game iterative improvement can only be obtained through long-term usage by users.

This study also has limitations in its research methodology. Although user needs and evaluations were incorporated in the fuzzy calculation process of establishing the evaluation rubric, there are shortcomings in the composition of the user sample data collected. The survey respondents were all users who had experienced CH-SGs, but many potential users were not surveyed. Additionally, the user groups need further segmentation, as different groups may have varying evaluations of serious games, leading to instability in results. The methods for collecting user needs should also be expanded beyond questionnaires, including in-depth interviews and group discussions for a more detailed understanding of user needs, thus improving the accuracy of the evaluation results. In the Delphi and AHP methods, the study included opinions from an expert panel, which was not fully comprehensive, being predominantly from the field of game studies. This led to subjectivity and bias in the evaluation system and weightings. The inclusion of experts from educational sciences, cultural heritage inheritors, game industry professionals, and other cultural institutions is needed for a more balanced perspective. Moreover, the indicators in the evaluation rubric are not fully comprehensive, with many derived from educational game evaluations. Regarding the design scales for evaluation indicators, this study used a 5-point rating scale, and the scoring of each indicator primarily relied on subjective user perceptions without more detailed criteria. Therefore, how to delineate evaluation levels more finely and completely is a topic for future discussion. Lastly, as the CH-SGs evaluation rubric developed in this study has significant cultural specificity within the serious games domain, whether it can serve as a referable and extendable evaluation paradigm for other types of serious games requires continuous validation in future research.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

PM: Writing – original draft. DC: Writing – review & editing.

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2024.1300686/full#supplementary-material

Cai, J. X. (2020). A brief discussion on the development of multimedia Technology in the Protection and Inheritance of intangible cultural heritage. Popular Liter. Art. 2020, 10–12.

Google Scholar

Caserman, P., Hoffmann, K., Müller, P., Schaub, M., Straßburg, K., Wiemeyer, J., et al. (2020). Quality criteria for serious games: serious part, game part, and balance. JMIR Serious Games 8:e19037. doi: 10.2196/19037

PubMed Abstract | Crossref Full Text | Google Scholar

Chang, K. (2015). Research on user experience of online social networking platforms based on needs and behaviors. Master’s thesis. Huazhong Normal University

Chen, Y. R. (2023). Use and satisfaction of Mobile online games - taking female users as an example. Master’s thesis. Southwest Jiaotong University.2023-12-01

Chen, Y. H., and Liu, Y. Y. (2019). Analysis on the communication path of socialist core values based on serious games [J]. Modern Communication (Journal of Communication University of China) . 41, 96–100. Available at: https://kns.cnki.net/kcms2/article/abstract?v=fCqJ37DMrBmO0j8ybP1wbOTkUhWQKOODHUFHq-VxLY04p21sd8h1mkMlzTiS-gL0FEWX3NRP1LjnvFgSHJqQVGxiyitJO47dKb5rh4CElgZ7X24ivdBkKO5DTWNnJL-RByq02Fuwfag=&uniplatform=NZKPT&language=CHS

Dai, Q. (2023). Exploration of constructing an evaluation system for elementary school calligraphy teaching using the Delphi method. Master’s thesis. Hainan Normal University

Davis, F. D. (1989). Technology acceptance model: TAM. Al-Suqri, MN, AS Al-Aufi: Information Seeking Behavior and Technology Adoption , 205–219.

Dong, Z. Z. (2013). Construction research on user experience evaluation system of educational games. Master’s thesis. Nanjing Normal University

Dong, Y. (2017). Research on the factors influencing the reaction intention of Mobile social network users to contact product recommendation information. Master’s thesis. Jilin University

Fang, Y. C. (2023). Research on the factors influencing the continued use intention of game live streaming platforms. Master’s thesis. Nanchang University

Gardner, H., and Hatch, T. (1989). Educational implications of the theory of multiple intelligences. Educ. Res. 18, 4–10. doi: 10.3102/0013189X018008004

Crossref Full Text | Google Scholar

Ge, Y. K., and Sun, Y. J. (2014). Quantitative analysis of the causes of credit risk in commercial banks - based on fuzzy analytic hierarchy process. Sci. Technol. Manag. 16, 106–109. doi: 10.16315/j.stm.2014.01.014

Gong, C. B., and Liang, P. P. (2018). From “user experience” to “media user experience” - an analysis of several basic issues on media user experience. Journal. Commun. Review 354, 67–74.

Holbrook, M. B., and Hirschman, E. C. (1982). The experiential aspects of consumption: consumer fantasies, feelings, and fun. J. Consum. Res. 9, 132–140. doi: 10.1086/208906

Hu, B. J. (2013). An empirical study on the relationship between customer perceived value, customer satisfaction, and customer loyalty in online games. Master’s thesis. South China University of Technology

Huang, Z. L. (2023). Research on the influence of user participation in Mobile game products from the perspective of use and gratification theory. Model. Simul. 12, 2505–2512. doi: 10.12677/MOS.2023.123230

Jia, J. P. (2019).Research on the continued acceptance intention of Mobile game users based on the expectation confirmation model. Master’s thesis. South China University of Technology

Jiang, Y. W. (2011). Analysis of business model innovation of online game operators. Master’s thesis. Southwest University of Finance and Economics

Jin, Y., and Yang, K. (2017). Research on the evaluation index system of information quality based on user experience: analysis from the perspective of user cognitive and emotional needs. Inform. Theory Pract. 40:5. doi: 10.16353/j.cnki.1000-7490.2017.02.019

Krathwohl, D. R., Bloom, B. S., and Masia, B. B. (1956). Taxonomy of educational objectives: the classification of educational goals. Affective Domain. 58–60.

Lei, Y. G., and Feng, M. T. (2020). Research on the design and development of educational games for preschool children from the perspective of learning quality. China Distance Educ. 2020:8.

Li, G. X. (2011). Research on the influencing factors of consumer behavior in online game products. Master’s thesis. Southwest University of Finance and Economics

Li, H. S. (2019). Design and Research of Functional Games of Intangible Cultural Heritage, Master Dissertation , Chongqing University. doi: 10.27670/d.cnki.gcqdu.2019.002378

Li, X. Y. (2023). From inside to outside: A study on the communication and shaping of the cultural IP of the Beijing palace museum. Master’s thesis. Shandong University.

Li, Y., and Deng, Q. Y. (2019). Analysis of the implementation effect of long-term care insurance system for the elderly based on fuzzy comprehensive evaluation method. Popul. Econ. 2019, 82–96. doi: 10.1007/978-3-319-94574-3

Li, W. H., and Guo, J. P. (2004). Interval weight of judgment matrix and its consistency test in AHP. J. Syst. Manag. 13, 530–532. doi: 10.3969/j.issn.1005-2542.2004.06.012

Li, F., and Sun, Y. (2019). Functional games: definition, value exploration, and development suggestions. Educ. Media Res. 2019:4.

Li, B. M., and Zhu, Z. T. (2017). From “learning to learn” focusing on results to “learning to learn” focusing on the process - research on online Learners’ Online learning ability assessment and development strategies. Open Educ. Res. 23, 92–100. doi: 10.13966/j.cnki.kfjyyj.2017.04.009

Liang, H. Z. (2019). Research on the comprehensive benefit evaluation of cultural and tourism characteristic towns based on AHP-fuzzy comprehensive evaluation method. Master’s thesis. Guangxi University

Lina, J., and Haifeng, W. (2020). Cultural value and interaction design of functional games - taking “Nishan Shaman” as an example [J]. J. Tonghua Normal Univ. 41, 1–7. doi: 10.13877/j.cnki.cn22-1284.2020.11.001

Liu, W. T., Gu, H., and Li, C. H. (2011). Expert evaluation method based on the Delphi method. Comput. Eng. 2011:4.

Liu, L., Liu, Q., and Yuan, Y. (2023). Learning player learning analysis model for serious games. Software Guide 22, 286–290.

Liu, W. H., Wang, Y. T., and Zhao, M. (2021). Design and development of evaluation indicators for educational games. 111–120.

Min, M. Y. (2018). Research on the continued usage intention of Mobile network game users based on marketing perspective. Master’s thesis. Wuhan University of Technology

Niu, Y. Y. (2020). An empirical study on the continued use intention of WeChat among college students based on the theory of use and satisfaction. Master’s thesis. Zhengzhou University

Peng, H. P., and Liu, N. N. (2020). Cultivating “play merchants” among minors in the web 3.0 era. Youth Develop. Forum 30:8

Qin, F. (2017). Research on the digital survival and development of intangible cultural heritage . University of Science and Technology of China, PhD dissertation. Available at: https://kns.cnki.net/kcms2/article/abstract?v=fCqJ37DMrBk-naE2hSj_xUlHtFubu4-XCtPfPhT-xubzlBcmc3Che1K9M6W8KGZHNJ7XYeKci5qpFVFTUUlXffb75mYTYOFYr9Rry099Lj8m0NllNfy3igK8zRrOo-Q5bUquR866Wlu7VlwyWfzKqg==uniplatform=NZKPTlanguage=CHS

Shen, J. (2015). Comparative study on evaluation index system of educational games in China and the United States. Master’s thesis. Hangzhou Normal University

Shen, J., and Zhang, S. J. (2014). Review of research on evaluation methods for educational games. J. Dist. Educ. 32:8. doi: 10.3969/j.issn.1672-0008.2014.03.015

Sun, F. (2023). Research on the influencing factors of intention to use massive open online course (MOOC)platforms. Master’s thesis. Hebei Normal University

Sun, J. J., and Guo, Y. Z. (2023). Research on the impact of digitalization of cultural heritage on National Heritage Responsibility: model construction based on TTF and TAM. Tourism Sci. 37, 71–85.

Sun, S. J., and Zhang, Y. H. (2017). Research on user participation motivation in social electronic commerce UGC platform: a case study of Xiaohongshu. Design 2017:2. doi: 10.3969/j.issn.1003-0069.2017.07.005

Tang, Z. (2022). Use and satisfaction: research on the motivation of Mobile internet entertainment user needs in the era of Mobile internet-taking “peace elite” as an example. News Vanguard. 2022:2.

Teng, C. E. (2019). Research on document protection of intangible cultural heritage from the perspective of social memory. Doctoral dissertation, Jilin University. Available at: https://kns.cnki.net/kcms2/article/abstract?v=fCqJ37DMrBnJxhInz-u7QSdlzh3qVbHdlVEw5fJxzEmN95O90eJvqjBFnNTNXanji5UOrqT3J_wmc1NuwVw1c-FnxIAGipIM72d4-ytJvQK77Hxg__8RBCkOjUN77ja4k2k9myAKHjl9PanMCkb2cQ==uniplatform=NZKPTlanguage=CHS

Wang, Y. W. (2010). Research on the factors influencing user participation intention in MMORPG online games. Master’s thesis. University of Electronic Science and Technology

Wang, L. (2017). Design and implementation of comprehensive ability evaluation system based on fuzzy analytic hierarchy process. China Manag. Inform. 2017, 184–186.

Wang, R. Q. (2022). On the application of cross-media narrative in game IP development - taking “Arknights” as an example. Writ. Edit. 2022, 16–17.

Wang, W. J., Zhao, X. C., and Xie, H. X. (2019a). New advances in the evaluation of foreign digital educational games. Compar. Educ. Res. 41, 101–108. doi: 10.20013/j.cnki.ice.2019.03.014

Wang, Y. M., Zhu, X. D., and Fu, S. S. (2019b). Empirical study on the factors influencing users’ intention to use mobile games based on immersion theory. Electron. Commerce 2019:4.

Wu, X. Y. (2019). Research on the factors influencing the stickiness of Mobile game users based on TAM and flow theory. Master’s thesis. Beijing Jiaotong University

Xu, F. (2009). Quality evaluation method of distance education based on AHP-FCE model. Master’s thesis. Jiangnan University

Xu, H. F. (2019). A study on the application of serious games in material cultural heritage Shenyang Aerospace University.

Xu, L. Q. (2019). User Experience Evaluation Model and Product Design Based on Cognitive Theory, Master Dissertation , Qilu University of Technology. Available at: https://kns.cnki.net/kcms2/article/abstract?v=fCqJ37DMrBkFBn-GK2LMhLsi8s-mH_V7Tqa6cI9EDPhzDOnBJN43f56xpNTv48ak7D8__LCZqNG1TB2h6YxSRO7JkAjhM8c0M7RY9FS_WJriTc2wer45cRNQk2rK2G7Pck_Q83cDqA5fUAMapaFRCw==uniplatform=NZKPTlanguage=CHS

Yang, X. (2020). Master Dissertation on Serious Games and Digital Protection of Intangible cultural Heritage, Huazhong University of Science and Technology . doi: 10.27157/d.cnki.ghzku.2020.001487

Yao, Z. F. (2023). Research on factors influencing the continued use intention of game-based WeChat Mini programs .

Ye, N., Zuo, B., and Zhang, L. (2009). A study on the relationship between motivation and addiction to online games among college students. Heilongjiang Higher Educ. Res. 2009:3. doi: 10.3969/j.issn.1003-2614.2009.04.031

Yu, X. Y. (2016). Construction of evaluation criteria for educational games. Curric. Teach. Res. 2016, 30–33.

Yu, Y. W., and Cheng, R. F. (2012). A comparative study on the cost of acquiring new customers and retaining old customers. Times Econ. Trade 2012:2.

Zhang, H. (2015). Research on the use value of virtual game products. Master’s thesis. Shanghai Jiao Tong University

Zhang, D. (2016). Analysis of factors leading to user churn in Mobile games. Master’s thesis. Nanjing University

Zhang, D. J. (2018). The artistry and Functionality of casual game interface design [J]. Oriental Collection . 269.

Zhang, W. M. (2019). Research on continuous use intention of MOBA mobile game users, Master Degree Thesis , Northwest University. Available at: https://kns.cnki.net/kcms2/article/abstract?v=fCqJ37DMrBk3qjzSH7e8cb_VEcbbt8FGXxDwD7xVL-G4LUKrV0kYIWv0v0l5qGQu_T1NeQVzY5 T47x9uqtZoG_HhTLaIEtV6wk3ifJHHMgtZuEHJIyGu2WP9iUKXXDvE0wTYAElmv611IYVBpx2JCA==uniplatform=NZKPTlanguage=CHS

Zhang, S. Y. (2023). Research on the Design of Serious Games for Changsha kiln cultural heritage VR on VIVE focus

Zhang, X. W. (2023). Research on the design index system of news APP products based on fuzzy analytic hierarchy process. Master’s thesis. Beijing University of Posts and Telecommunications.

Zhang, K. Y., Mo, Q. Y., and Li, R. J. (2014). Research on the impact of driving safety based on AHP method. Sci. Technol. Innov. Appl.. 2014:2.

Zhong, Z. J. (2011). Basic characteristics of online game players and socialization behavior in games. Modern Commun. 2011:5.

Zhou, Y., and Xu, X. (2017). Review of digitalization research on intangible cultural heritage. Library Inform. Serv. 1, 6–15. doi: 10.13266/j.issn.0252-3116.2017.02.001

Zhu, Y. H., Xu, S. Y., and Wu, J. F. (2020). Research on the evaluation model of ride-hailing service experience based on analytic hierarchy process. Packaging. Engineering 41, 112–117. doi: 10.19554/j.cnki.1001-3563.2020.14.016

Keywords: serious games, cultural heritage, continuous usage intention, evaluation rubric, Delphi method, analytic hierarchy process, fuzzy comprehensive evaluation method

Citation: Mao P and Cho DM (2024) Research on an evaluation rubric for promoting user’s continuous usage intention: a case study of serious games for Chinese cultural heritage. Front. Psychol . 15:1300686. doi: 10.3389/fpsyg.2024.1300686

Received: 17 October 2023; Accepted: 15 January 2024; Published: 15 February 2024.

Reviewed by:

Copyright © 2024 Mao and Cho. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Pingting Mao, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

  • Search Menu
  • Volume 2024, Issue 2, February 2024 (In Progress)
  • Volume 2024, Issue 1, January 2024
  • Case of the Year
  • MSF Case Reports
  • Audiovestibular medicine
  • Cardiology and cardiovascular systems
  • Critical care medicine
  • Dermatology
  • Emergency medicine
  • Endocrinology and metabolism
  • Gastroenterology and hepatology
  • Geriatrics and gerontology
  • Haematology
  • Infectious diseases and tropical medicine
  • Medical ophthalmology
  • Medical disorders in pregnancy
  • Paediatrics
  • Palliative medicine
  • Pharmacology and pharmacy
  • Radiology, nuclear medicine, and medical imaging
  • Respiratory disorders
  • Rheumatology
  • Sexual and reproductive health
  • Sports medicine
  • Substance abuse
  • Author Guidelines
  • Submission Site
  • Open Access
  • Editorial Board
  • Advertising and Corporate Services
  • Journals Career Network
  • Self-Archiving Policy
  • Journals on Oxford Academic
  • Books on Oxford Academic

Issue Cover

Article Contents

Introduction, case presentation, conflict of interest statement, ethical approval, consent for publication.

  • < Previous

Patient with a history of Glanzmann thrombasthenia presented with chronic subdural hematoma: a case report study

  • Article contents
  • Figures & tables
  • Supplementary Data

Mohammad Hasanpour, Hajar Mehdizadeh, Patient with a history of Glanzmann thrombasthenia presented with chronic subdural hematoma: a case report study, Oxford Medical Case Reports , Volume 2024, Issue 2, February 2024, omae004, https://doi.org/10.1093/omcr/omae004

  • Permissions Icon Permissions

Glanzmann thrombasthenia (GT) is a rare platelet disorder characterized by qualitative/quantitative deficiencies of the platelets’ fibrinogen receptor, glycoprotein (GP) IIb/IIIa complex, resulting in impaired platelet aggregation and increased bleeding time. Most cases are hereditary with an autosomal recessive pattern of inheritance, but acquired GT also occurs. We report the surgical management of symptomatic chronic subdural hematoma (CSDH), a rare condition in young individuals, in a 37-year-old man who had GT and a history of mild head trauma approximately one month before admission. Despite hematologic consultation, normal bleeding time and clotting time, and platelet transfusion before surgery, massive hemorrhage during surgery, epidural hematoma, and anisocoria in the ICU occurred that led to craniectomy. This report highlights that CSDH management in patients with GT requires close monitoring of these patients as well as collaboration between neurosurgeons, intensive care physicians, hematologists, and anesthesiologists.

Glanzmann thrombasthenia (GT) is a rare bleeding disorder featured with qualitative or quantitative deficiencies of a fibrinogen receptor (glycoprotein (GP) IIb/IIIa) on the platelets, leading to absence of/impaired platelet aggregation, decreased clot retraction, and increased bleeding time while the platelet count is low to normal [ 1 ]. Although GT is mostly an inherited autosomal recessive disorder, acquired GT may also occur due to auto-immunization [ 1 ]. GT has a prevalence of one per million individuals and is more frequent in countries with a higher incidence of consanguinity [ 2 ].

Chronic subdural hematomas (CSDH) are liquefied hematomas in the subdural space with a characteristic outer membrane that occurs a few weeks after head injury and one of its risk factors is thrombopathy [ 3 ]. CSDH are typically seen in the elderly population and are rare in young adults [ 4 ]. The most common theory for the cause of CSDH is that minor inertial brain injury causes movement of the brain within the skull and tears bridging veins as they traverse the cell layer of the dural border [ 3 ]. Without timely interventions, CSDH may be fatal despite their benign nature [ 4 ]. It is consensus that patients with symptoms that can be attributed to radiologically confirmed CSDH should be treated [ 3 ].

Platelet transfusion is the standard treatment for patients with GT as a countermeasure for major surgeries. Recombinant activated factor VII (rFVIIa) has also been used in patients with antiplatelet antibodies that render transfusion ineffective [ 5 ]. A lack of scientific knowledge of rare diseases and their appropriate management may result in poor treatment outcomes. Thus, this study aimed to report the case of a young male with GT who had symptomatic CSDH.

A 37-year-old man was admitted to the emergency department of Taleghani Hospital (Kermanshah, Iran) with complaints of intractable new onset headache, nausea, vomiting, and confusion. Approximately 4 weeks before admission, he had a mild head trauma. Past medical history was remarkable for opioid abuse, intermittent spontaneous otorrhagia controlled by repeated platelet transfusion and GT. However, his laboratory data was unremarkable.

On the examination at admission, he showed normal vital signs. Although the patient’s score on the Glasgow Coma Scale (GCS) was 13/15, no other remarkable abnormalities were found by neurological examination. Due to impaired consciousness and a history of head injury, the patient underwent a brain CT scan that revealed a significant hypo-dense left frontoparietal hematoma and the CSDH were diagnosed. Thus, a burr-hole craniostomy was performed.

Although the preoperative testings showed normal bleeding time (BT) and clotting time (CT), the patient received 5 units of platelet preoperatively after consultation with a hematologist. Surgery was started with two burr holes in the frontoparietal area. The hematoma was evacuated but massive and uncontrollable hemorrhage was seen through the bone and scalp. As possible, partial hemostasis was achieved by bone wax and surgical hemostasis dressing (surgicel) and bipolar cautery, drains were fixed, and the scalp was stitched.

Then, the patient was admitted to the intensive care unit (ICU) by concomitant packed cell transfusion. However, his GCS score began to decrease and he became anisocoric. Immediately, red blood cell and platelet transfusion was done and reoperation was conducted by craniectomy. The epidural and subdural hematoma was evacuated and hemostasis was acceptably achieved by surgicel, bone wax, and bipolar. The dural tack-up stitch was done, the drain was fixed and the scalp was stitched by continuous vertical mattress technique ( Fig. 1 ). Due to the possibility of massive hemorrhage during subcutaneous bone flap storage, the bone flap was discarded. During the craniectomy and until 48 h, the patient was infused with intravenous tranexamic acid 2 gr every 6 h and fibrinogen 1 gr every 8 h. He spent 5 days in the ICU and 5 days on the ward before discharge. From the first surgery until discharge, he received 35 units of pooled platelet, 7 packed cells, and 4 FFP.

CT scan after the second operation.

CT scan after the second operation.

One month later, he was admitted because of a headache and the CT scan showed epidural hematomas ( Fig. 2 ). Hence, he was taken to the operating room for hematoma evacuation. In the laboratory examination, increased BT (10 min) was found, and 2 units P.C., 2 units single donor platelet (SDP), and 33 units random donor platelet (RDP) were transfused preoperatively, leading to decreased BT (4 min). After surgery, he received 2 units P.C., 2 units platelet, and 5 units RDP and was discharged home after 5 days.

CT scan before the third operation.

CT scan before the third operation.

Two months later, he was admitted for cranioplasty, and a hematology and anesthesiology consult was done. BT and CT were normal but perioperatively he received 35 units RDP, 2 units SDPs, and 5 units P.C. Despite a huge platelet transfusion before and after cranioplasty, bleeding in the ward was reported to the neurosurgery resident by the nurse. It wasn’t controlled by overrunning stitches because of bleeding in the needle insertion sites. Finally, bleeding was controlled by tight elastic bandages. After 5 days, the patient was discharged. On follow-up at 2 months, he had no neurologic deficit or residual symptoms.

GT is an extremely rare inherited disorder of platelet function with an autosomal recessive mode of inheritance. However, acquired GT due to autoimmunization has also been reported. GT is characterized by qualitative/quantitative deficits of a platelet’s fibrinogen receptor, GP IIb/IIIa, resulting in impaired platelet aggregation and increased bleeding time [ 1 ]. This rare disorder is more common in populations with a higher rate of consanguinity [ 2 ]. Common clinical manifestations of GT include spontaneous mucosal and skin bleeding [ 6 ]. There are three subtypes for GT, but no correlation has been found between the bleeding severity and the GT subtype [ 7 ].

No therapy is required in patients with GT regularly. However, treatment is always required for these patients during spontaneous bleeding episodes as well as surgical procedures, which are significant challenges due to bleeding probability and a high incidence of alloimmunization induced by repeated platelet transfusion [ 1 , 7 ]. Local measures (e.g. local pressure, ice therapy, topical thrombin, fibrin sealants, etc.) and/or antifibrinolytic agents (e.g. tranexamic acid) are usually effective in the case of mild to moderate bleeding episodes. Platelet transfusion is the standard treatment for moderate to severe bleeding and the routine prophylaxis for major surgeries in patients with GT [ 1 ]. Alternative treatment with rFVIIa has also been recommended for patients who are alloimmunized, but it is expensive and associated with the risk of thrombosis [ 1 , 7 , 8 ].

CSDH are usually caused by slow bleeding from bridging veins on the cerebral surface. However, its pathophysiology has not yet well been understood. While CSDH is one of the most common cases of neurosurgery, it is usually seen in elderly subjects and is rare in young individuals [ 3 ]. Its management is based on the presence and severity of hematoma-related symptoms and hematoma size and mass effect. Surgery is required for symptomatic CSDH, hematoma thickness > 10 mm, or midline shift > 7 mm [ 9 ]. The burr-hole craniostomy with/without a drainage system is the most common and most efficient surgical technique used for CSDH management. In the case of hematoma recurrence, burr-hole conversion into a craniotomy should be considered [ 9 ]. CSDH is a benign condition, nonetheless, it may be fatal without prompt surgical intervention and hematoma recurrence is still an important concern [ 4 ].

To the best of the authors’ knowledge, there are no case reports of CSDH management in young patients with GT. As mentioned previously, the classical measures were done in our case, but simple neurosurgery for hematoma evacuation became complicated and nearly fatal despite the teamwork of the neurosurgeon, hematologist, and anesthesiologist. Unfortunately, there are no established recommendations for treating and prophylaxis of surgical bleeding in patients with GT and a combination of functional platelet tests and clinical assessment is used for assessing the response to treatment [ 10 ]. However, a useful management algorithm has been proposed by Solh et al. for elective and urgent surgical interventions in these patients [ 1 ].

We reported a rare case of a young patient with a CSDH who also had rare GT and a history of repeated platelet transfusion to control spontaneous otorrhagia. This report highlights the need for close monitoring in patients with GT when neurosurgery is done for CSDH management. One important point is that despite having normal BT and CT, the patient’s bleeding during and after the surgery was difficult to control. Therefore, as previous studies have also shown, BT and CT are poor predictors of clinical bleeding. In light of our experience with this case, we suggest if a patient with CSDH and GT is neurologically and hemodynamically stable, the neurosurgeons should be more prudent and perform more investigations as needed, like what was suggested by Solh et al. Further, close collaboration between neurosurgeons, intensive care physicians, hematologists, and anesthesiologists is critical for CSDH management in patients with GT.

None declared.

There was no funding for this study.

No ethical approval was needed for this case report.

The patient gave written informed consent for the publication of this report.

Solh   T , Botsford   A , Solh   M . Glanzmann’s thrombasthenia: pathogenesis, diagnosis, and current and emerging treatment options . J Blood Med   2015 ; 6 : 219 – 27 .

Google Scholar

Mohan   G , Malayala   SV , Mehta   P , Balla   M . A comprehensive review of congenital platelet disorders, thrombocytopenias and thrombocytopathies . Cureus   2020 ; 12 :5–6. https://doi.org/10.7759/cureus.11275 .

Winn   HR . Youmans and Winn Neurological Surgery E-Book: 4-Volume Set . Amsterdam, Netherlands: Elsevier Health Sciences , 2022 .

Google Preview

Won   YD , Yi   H-J , Lee   YJ , Chun   H-J , Cho   H , Bak   K-H . Chronic subdural hematoma in young adult: an age comparison study . Korean J Nutr   2013 ; 9 : 6 – 11 .

Rajpurkar   M , Chitlur   M , Recht   M , Cooper   DL . Use of recombinant activated factor VII in patients with Glanzmann's thrombasthenia: a review of the literature . Haemophilia   2014 ; 20 : 464 – 71 .

Lee   SM , Kim   KN , Kim   SY . Perioperative hemostatic management of a pediatric patient with Glanzmann thrombasthenia undergoing osteoplastic craniotomy and hematoma removal: a case report . Acta Haematol   2019 ; 142 : 244 – 8 .

Ganapule   A , Jain   P , Abubacker   FN , Korula   A , Abraham   A , Mammen   J . et al.    Surgical procedures in patients with Glanzmann's thrombasthenia: case series and literature review . Blood Coagul Fibrinolysis   2017 ; 28 : 171 – 5 .

Kumar   N , Khan   AA , Kumar   N , Bindra   A , Mahajan   C , Goyal   K . Glanzmann thrombasthenia and its perioperative management in head trauma: a rare and challenging situation . J Neuroanaesth Crit. Care   2016 ; 3 : 052 – 5 .

Nouri   A , Gondar   R , Schaller   K , Meling   T . Chronic subdural hematoma (cSDH): a review of the current state of the art . Brain Spine   2021 ; 1 : 100300 .

Jayakrishnan   TT , Limonnik   V , Shah   D , Mewawalla   P . Glanzmann’s thrombasthenia: how listening to the patient is sometimes the simple key to good medicine!   Case Rep Med   2020 ; 2020 : 1 – 3 .

Supplementary data

Email alerts, citing articles via, affiliations.

  • Online ISSN 2053-8855
  • Copyright © 2024 Oxford University Press
  • About Oxford Academic
  • Publish journals with us
  • University press partners
  • What we publish
  • New features  
  • Open access
  • Institutional account management
  • Rights and permissions
  • Get help with access
  • Accessibility
  • Advertising
  • Media enquiries
  • Oxford University Press
  • Oxford Languages
  • University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

  • Copyright © 2024 Oxford University Press
  • Cookie settings
  • Cookie policy
  • Privacy policy
  • Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

IMAGES

  1. Case Study Rubric

    case study discussion rubric

  2. Rubrics

    case study discussion rubric

  3. case study rubric.pdf

    case study discussion rubric

  4. Case Study Rubric

    case study discussion rubric

  5. Case Study Grading Rubric

    case study discussion rubric

  6. Case Study Grading Rubric(1)

    case study discussion rubric

VIDEO

  1. Basic study of rubric and repertory In Hindi

  2. Virtual CPE Programme on Code of Ethics Case Study Discussion

  3. Introduction to Case Study & Discussion on Pre Seen :10 12 2022

  4. Case Study Rubric Paper

  5. CPE Programme on Code of Ethics Case Study Discussion

  6. vanilla case study by mind rubric

COMMENTS

  1. PDF Case Study Grading Rubric Presenter Name: Judges Initials: 4 3 2 1 0 Score

    features of the case • identifies some problems in the case The presenter(s): • discusses few of the important aspects of the background of the case not demonstrate unique • demonstrates few unique features of the case • identifies a few problems in the case The presenter(s) does not ts, problems of the case and do feature of the case.

  2. PDF Sample rubric for online discussions

    Sample Online Discussions Rubric: This rubric is intended to assess students' participation on asynchronous online discussion forums. The criteria listed below can be used to create a rubric on ICON. Having a rubric promotes consistent grading and clarity of coursework expectations.

  3. PDF Discussion Rubric Examples

    Online Discussion Rubric ** Open class discussion is an important and significant part of an online course. While class discussion whether online or face to face, can be characterized by free flowing conversation, there are identifiable characteristics that distinguish exemplary contributions to class discussion from those of lesser quality.

  4. PDF Using a Rubric to Evaluate Quality in Case Study Writing

    In a rubric, descriptions under each trait are not meant as a checklist but are followed by indicators of performance level.8 For each of the elements in this rubric, there are four levels of performance: 1) unacceptable, 2) improvement required, 3) professional level, and 4) exemplary.

  5. Do Your Students Know How to Analyze a Case—Really?

    Step 1: Problem definition. What is the major challenge, problem, opportunity, or decision that has to be made? If there is more than one problem, choose the most important one. Often when solving the key problem, other issues will surface and be addressed.

  6. Case Study

    Using a rubric to assess student performance using a case study in an online discuss to facilitate investigation and research of content. Rubric example: a case study See Also CTLM Instructional Resources

  7. PDF Discussion Summary and Case Study Grading Rubric (10 pts)

    Discussion Summary and Case Study Grading Rubric (10 pts) Excellent (100%) Good (90%) Decent (80%) Needs Work (70% or below) Did you follow the instruction? (2 pts) All of them. Missed one. ... Is your case study relevant to the discussion? Are the three questions appropriate? (4 pts) Relevant case study AND all the questions are appropriate.

  8. PDF Case Study Rubric

    Review Design: The review rubric is designed to allow you to quantify your responses to various features of the manuscript and provide notes to support your responses. As such: If you. highlight highlight. 3, 2, or 1, identify what is absent in the section of the manuscript. If you 6, 5, or 4, identify ways to strengthen that section in ways ...

  9. PDF Case Study Evaluation Rubric

    The National School Psychology Certification Board (NSPCB) of the National Association of School Psychologists (NASP) developed the following rubric to help guide applicants in structuring an effective case study. Additionally, the NSPCB utilizes the rubric as part of the evaluation process for NCSP candidates from graduate programs without ...

  10. PDF Case Study Rubric

    IEEE TPC Review Rubric for Case Studies 1 Rev. 06/24/21 Review Rubric for Case Studies Reviewers: Please complete and return this form when reviewing a manuscript. If you have annotated the manuscript, please return that file with this form. Review Design: The review rubric is designed to allow you to quantify your responses to

  11. Assessment for Curricular Improvement

    Guidelines for leading classroom discussion of case studies. 3. Case studies. Case studies should be complex enough and realistic enough to be challenging, yet be manageable within the time frame. ... and e) tolerance for ambiguity. Scoring rubrics for ethics case analyses should address these outcomes, not basic knowledge of the ethical ...

  12. Create Discussion Rubrics

    Online Discussion Rubric - This rubric was used in a series of small group discussion activities on related module topics in a world literature course, Farrah Cato, Instructor in UCF's Department of English at the College of the Arts & Humanities.

  13. PDF The Online Discussion Conference Grading Rubric

    Sample Grading Rubric for Online Discussion - Middle Tennessee State - Example 1. Class participation is an important expectation of this course. Students are expected to offer comments, questions, and replies to the discussion question that have been posed for each module as well as to classmate postings.

  14. Multiple Rubric-based Assessments of Student Case Presentations

    We describe a pilot study and implementation of a grading rubric that facilitates assessment of student case presentations from 3 sources: instructors, peers, and self. The objectives of the pilot study were to (1) determine the difference between checklist and rubric-based instructor scores; (2) determine the difference between composite ...

  15. A Rubric for Evaluating Student Analyses of Business Cases

    Case analysis rubric. ... The undergraduate case research study model. Journal of Management Education, 34, 574-604. Crossref. Google Scholar. White C. S. (2007). Levels of understanding: A guide to the teaching and assessment of knowledge. Journal of Business Education, 82, 159-163.

  16. Create a Case Method Group Activity to Engage Students in Critical

    Individual Paper Case Study Rubric. Case Study Rubric for Group Collaboration Discussion. Example #2. Instructor: Revathi Viswanathan Course: Biotechnology. Students were asked to discuss case studies relating to their subject, and Edmodo was used as a learning platform for handling them.

  17. iRubric: Case Study rubric

    Case Study Case Study Applies to written case reports Rubric Code: C227279 By dawnwickstrom Ready to use Public Rubric Subject: Business Type: Assessment Grade Levels: Undergraduate Keywords: Case study rubric Subjects: Business Finance Types: Assessment Discuss this rubric You may also be interested in: More rubrics by this author

  18. Using Rubrics and Content Analysis for Evaluating Online Discussion: a

    This paper presents a case study of using course-specific rubrics combined with content analysis, together with instructor and student feedback, to assess learning via online discussion.

  19. Case Study Discussion Grading Rubric Guidelines & Scoring Rubric

    Chamberlain College of Nursing NR 535Theoretical Foundations and Instructional Strategies for the Nurse Educator Case Study Discussion Grading Rubric Guidelines & Scoring Rubric Purpose The purpose of this assignment is to provide an opportunity for learners to: (a) articular key questions to be asked in a grievance; (b) review an academic ...

  20. Module One Case Study Guidelines and Rubric (html)

    ACC 692 Module One Case Study Guidelines and Rubric Overview Rita Crundwell operated the largest municipal fraud in U.S. history by embezzling $53 million. In this case study analysis, you will analyze the case by discussing what she did and why she was able to carry out this fraud for almost two decades. Prompt View the video Rita Crundwell from Dixon, Illinois.

  21. PDF Case Analysis Grading Rubric

    15-21 pts. Throughout little to none of the work. 0-7 pts. Support 35 pts. claims are supported with detailed and persuasive examples. the analysis incorporates required and additional resources, when necessary. Throughout the. whole work. 29-35 pts.

  22. Using Rubrics and Content Analysis for Evaluating Online Discussion: a

    This paper presents a case study of using course-specific rubrics combined with content analysis, together with instructor and student feedback, to assess learning via online discussion. Student feedback was gathered via Small Group Instructional Diagnosis, and instructor feedback was collected through formal interviews. Content analysis used emergent coding with different assessment criteria ...

  23. Case study presentation rubric

    Case Study Presentation Grading Rubric. Requiremen t. 3 2 1. Content. A, B. Includes all components: describes news story about a case, identifies source & defines legal/ethical issues. Missing one component or partially describes each component. Missing two or more components or does not fully describe each component. C

  24. Frontiers

    This study posits that with the guidance of evaluation rubrics, the design process of CH-SGs is akin to having a measuring ruler and a direction, leading to breakthroughs in research on design, development, and application evaluations of CH-SGs (Wang et al., 2019a). An evaluation rubric is a scoring tool that includes standards for assessing a ...

  25. Case Study and Discussion Post Rubric Intro to Bus

    Introduction to Business Barrow/BUSI 1030 Case Study Analysis and Discussion Rubric Case Study Analysis and Discussion Rubric Clarity & Structure The writer's response is well-organized, concise, and readable. The writer presents ideas in a logical, cohesive sequence. Integration of Knowledge & Support Articles The writer supports claims ...

  26. Health 305

    Category Unacceptable (0-1) Needs Improvement (2-3) Good (4) Excellent (5) Total Possible Points; Financial Impact Analysis (x2) Analysis lacks depth and contains significant inaccuracies.

  27. Patient with a history of Glanzmann thrombasthenia presented with

    INTRODUCTION. Glanzmann thrombasthenia (GT) is a rare bleeding disorder featured with qualitative or quantitative deficiencies of a fibrinogen receptor (glycoprotein (GP) IIb/IIIa) on the platelets, leading to absence of/impaired platelet aggregation, decreased clot retraction, and increased bleeding time while the platelet count is low to normal []. ...