publications | Hung K. Chau

2023

Explainable Course Recommendation: Connecting College Education to Knowledge and Careers Through Skills

Hung Chau

University of Pittsburgh, 2023

Abs HTML

Academic choice and exploration are essential aspects of undergraduate education in the United States, allowing students to select courses with minimal restrictions. However, students often face challenges in navigating the complex academic landscape, hindered by limited information, insufficient guidance, and an overwhelming number of choices. Time constraints from the academic calendar and high demand for popular courses make a thorough evaluation of options difficult. Although academic institutions provide career guidance counselors or advisers, the number of advisers is still limited. Course recommendation systems aim to offer personalized suggestions based on students’ academic backgrounds, preferences, skills, and career goals. However, there is a lack of research on students’ perceptions of recommendations and the provision of explanations to help them evaluate course relevance. Moreover, the majority of course recommender systems concentrate only on the context of learning in higher education. Despite the importance of career goals, none have attempted to establish a connection between learning and work by incorporating job information into course recommendation and explanation. This dissertation explores the development of an advanced course recommendation system in higher education, linking academic courses to career paths using deep learning and natural language processing techniques. It begins by examining various methods for representing and recommending courses, utilizing institutional big data and combining content-based and collaborative models for better performance. A central aspect of this dissertation is the development of skill-based explanations for course recommendations. This is achieved by employing a deep concept extraction model that utilizes BERT and BI-LSTM-CRF architectures. This model effectively extracts concepts from course descriptions, enhancing the recommendation process. Using this concept extraction model, I investigate the impact of skill-based explanations in a serendipitous course recommendation system, which was tested using the AskOski system at the University of California, Berkeley. The findings indicate that these explanations not only increase user interest, particularly in courses with high unexpectedness, but also bolster decision-making confidence. To achieve greater personalization in course recommendations, the future of this field extends beyond academics by incorporating insights from the job market to align with students’ career aspirations. This dissertation introduces an innovative approach for integrating skill-related data into course recommendation systems, effectively bridging the gap between academic pursuits and career aspirations. Finally, I develop an explainable, personalized course recommendation system that incorporates insights from the job market. This system tailors course suggestions based on students’ academic histories and career preferences. Its objective is to enhance course exploration in higher education, assisting students in navigating their educational paths and acquiring the essential skills required for their chosen majors and future careers. A user study conducted at the University of Pittsburgh demonstrates that the recommendations were generally perceived as valuable, with explanations playing a pivotal role in aiding students to assess their interest in the recommended courses. This underscores the significance of integrating skill-related data and explanations into educational recommendation systems.
Connecting higher education to workplace activities and earnings

Hung Chau , Sarah H Bana , Baptiste Bouvier , and 1 more author

PloS one, 2023

Abs HTML

Higher education is a source of skill acquisition for many middle- and high-skilled jobs. But what specific skills do universities impart on students to prepare them for desirable careers? In this study, we analyze a large novel corpora of over one million syllabi from over eight hundred bachelors’ granting US educational institutions to connect material taught in higher education to the detailed work activities in the US economy as reported by the US Department of Labor. First, we show how differences in taught skills both within and between college majors correspond to earnings differences of recent graduates. Further, we use the co-occurrence of taught skills across all of academia to predict the skills that will be taught in a major moving forward. Our unified information system connecting workplace skills to the skills taught during higher education can improve the workforce development of high-skilled workers, inform educational programs of future trends, and enable employers to quantify the skills of potential workers.
Knowledge Annotation for Intelligent Textbooks

Mengdi* Wang , Hung* Chau , Khushboo Thaker , and 2 more authors

Technology knowledge and learning, 2023

Abs HTML

With the increased popularity of electronic textbooks, there is a growing interest in developing a new generation of “intelligent textbooks,” which have the ability to guide readers according to their learning goals and current knowledge. Intelligent textbooks extend regular textbooks by integrating machine-manipulable knowledge, and the most popular type of integrated knowledge is a list of relevant concepts mentioned in the textbooks. With these concepts, multiple intelligent operations, such as content linking, content recommendation, or student modeling, can be performed. However, existing automatic keyphrase extraction methods, even supervised ones, cannot deliver sufficient accuracy to be practically useful in this task. Manual annotation by experts has been demonstrated to be a preferred approach for producing high-quality labeled data for training supervised models. However, most researchers in the education domain still consider the concept annotation process as an ad-hoc activity rather than a carefully executed task, which can result in low-quality annotated data. Using the annotation of concepts for the Introduction to Information Retrieval textbook as a case study, this paper presents a knowledge engineering method to obtain reliable concept annotations. As demonstrated by the data we collected, the inter-annotator agreement gradually increased along with our procedure, and the concept annotations we produced led to better results in document linking and student modeling tasks. The contributions of our work include a validated knowledge engineering procedure, a codebook for technical concept annotation, and a set of concept annotations for the target textbook, which could be used as a gold standard in further intelligent textbook research.

2021

Automatic Concept Extraction for Domain and Student Modeling in Adaptive Textbooks

Hung Chau , Igor Labutov , Khushboo Thaker , and 2 more authors

International Journal of Artificial Intelligence in Education, 2021

Abs HTML

The increasing popularity of digital textbooks as a new learning media has resulted in a growing interest in developing a new generation of adaptive textbooks that can help readers to learn better through adapting to the readers’ learning goals and the current state of knowledge. These adaptive textbooks are most frequently powered by internal knowledge models, which associate a list of unique domain knowledge concepts with each section of the textbook. With this kind of concept-level knowledge representation, a number of intelligent operations could be performed, which include student modeling, adaptive navigation support, and content recommendation. However, manual indexing of each textbook section with concepts is challenging, time-consuming, and prone to errors. Modern research in the area of natural language processing offers an attractive alternative, called automatic keyphrase extraction. While a range of keyphrase and concept extraction methods have been developed over the last twenty years, few of the known approaches were applied and evaluated in a textbook context. In this paper, we present FACE, a supervised feature-based machine learning method for automatic concept extractions from digital textbooks. This method has been created for building domain and student models that form the core of intelligent textbooks. We evaluated FACE on a newly constructed full-scale dataset by assessing how well it approximates concept annotations produced by human experts and how well it supports the needs of student modeling. The results show that FACE outperforms several state-of-the-art keyphrase extraction methods.
Orienting Students to Course Recommendations Using Three Types of Explanation

Run Yu , Zach Pardos , Hung Chau , and 1 more author

In Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization , 2021

Abs HTML

An emerging challenge in course recommender systems is explaining to students why they have been recommended particular courses. In the context of a university, it can be valuable for a recommender system to introduce students to courses they may have not otherwise taken but which are still relevant to them. However, there is a tension between these goals as students have less ability to judge the relevance of courses, the less familiar they are with them. In this paper, we explore ways of familiarizing students with recommendations with three types of explanations designed with varying levels of personalization. We conduct a 67 student randomized controlled experiment using two course recommendation engines, content and context-based, and augment recommendations with three different types of explanation. Students rated each course recommended in terms of novelty, unexpectedness, and successfulness (i.e., intent to enroll). We find several statistically significant results, including an increase in serendipity (i.e., unexpectedness + successfulness) when explaining new course recommendations using keywords from courses a student has previously taken.

2020

Understanding the Tradeoff between Cost and Quality of Expert Annotations for Keyphrase Extraction

Hung Chau , Saeid Balaneshin , Kai Liu , and 1 more author

In Proceedings of the 14th Linguistic Annotation Workshop , Dec 2020

Abs HTML

Generating expert ground truth annotations of documents can be a very expensive process. However, such annotations are essential for training domain-specific keyphrase extraction models, especially when utilizing data-intensive deep learning models in unique domains such as real-estate. Therefore, it is critical to optimize the manual annotation process to maximize the quality of the annotations while minimizing the cost of manual labor. To address this need, we explore multiple annotation strategies including self-review and peer- review as well as various methods of resolving annotator disagreements. We evaluate these annotation strategies with respect to their cost and on the task of learning keyphrase extraction models applied with an experimental dataset in the real-estate domain. The results demonstrate that different annotation strategies should be considered depending on specific metrics such as precision and recall.

2019

Data-Assistive Course-to-Course Articulation Using Machine Translation (**Best Paper Award**)

Zachary A. Pardos , Hung Chau , and Haocheng Zhao

In Proceedings of the Sixth (2019) ACM Conference on Learning @ Scale , Dec 2019

Article No.: 22

Abs HTML

Higher education at scale, such as in the California public post-secondary system, has promoted upward socioeconomic mobility by supporting student transfer from 2-year community colleges to 4-year degree granting universities. Among the barriers to transfer is earning enough credit at 2-year institutions that qualify for the transfer credit required by 4-year degree programs. Defining which course at one institution will count as credit for an equivalent course at another institution is called course articulation, and it is an intractable task when attempting to manually articulate every set of courses at every institution with one another. In this paper, we present a methodology towards making tractable this process of defining and maintaining articulations by leveraging the information contained within historic enrollment patterns and course catalog descriptions. We provide a proof-of-concept analysis using data from a 4-year and 2-year institution to predict articulation pairs between them, produced from machine translation models and validated by a set of 65 institutionally pre-established course-to-course articulations. Finally, we create a report of proposed articulations for consumption by the institutions and close with a discussion of limitations and the challenges to adoption.

2018

Course-Adaptive Content Recommender for Course Authoring

Hung Chau , Jordan Barria-Pineda , and Peter Brusilovsky

In Proceedings of the 14th European Conference on Technology Enhanced Learning (EC-TEL) , Dec 2018

Abs HTML

Developing online courses is a complex and time-consuming process that involves organizing a course into a sequence of topics and allocating the appropriate learning content within each topic. This task is especially difficult in complex domains like programming, due to the incremental nature of programming knowledge, where new topics extensively build upon domain concepts that were introduced in earlier lessons. In this paper, we propose a course-adaptive content-based recommender system that assists course authors and instructors in selecting the most relevant learning material for each course topic. The recommender system adapts to the deep prerequisite structure of the course as envisioned by a specific instructor, while unobtrusively deducing that structure from problem-solving examples that the instructor uses to present course concepts. We assessed the quality of recommendations and examined several aspects of the recommendation process by using three datasets collected from two different courses. While the presented recommender system was built for the domain of introductory programming, our course-adaptive recommendation approach could be used in a variety of other domains.
Learning Content Recommender System for Instructors of Programming Courses

Hung Chau , Jordan Barria-Pineda , and Peter Brusilovsky

In Artificial Intelligence in Education , Dec 2018

Abs HTML

In this paper, we present a course-adaptive recommender system that assists instructors of programming courses in selecting the most relevant learning materials. The recommender system deduces the envisioned structure of a specific course using program examples prepared by the course instructor and recommends learning content items adapting to instructor’s intentions. We also present a study that assessed the quality of recommendations using datasets collected from different courses.

2017

Content Wizard: Concept-Based Recommender System for Instructors of Programming Courses

Hung Chau , Jordan Barria-Pineda , and Peter Brusilovsky

In Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization , Dec 2017

Abs HTML

Authoring an adaptive educational system is a complex process that involves allocating a large range of educational content within a fixed sequence of units. In this paper, we describe Content Wizard, a concept-based recommender system for recommending learning materials that meet the instructor’s pedagogical goals during the creation of an online programming course. Here, the instructors are asked to provide a set of code examples that jointly reflect the learning goals that are associated with each course unit. The Wizard is built on top of our course-authoring tool, and it helps to decrease the time instructors spend on the task and to maintain the coherence of the sequential structure of the course. It also provides instructors with additional information to identify content that might be not appropriate for the unit they are creating. We conducted an off-line study with data collected from an introductory Java course previously taught at the University of Pittsburgh in order to evaluate both the practicality and effectiveness of the system. We found that the proposed recommendation’s performance is relatively close to the teacher’s expectation in creating a computer-based adaptive course.