publications

2023

1. 

   Connecting higher education to workplace activities and earnings

   Hung Chau ,  Sarah H Bana ,  Baptiste Bouvier , and 1 more author

   PloS one, 2023

   Abs HTML

   Higher education is a source of skill acquisition for many middle- and high-skilled jobs. But what specific skills do universities impart on students to prepare them for desirable careers? In this study, we analyze a large novel corpora of over one million syllabi from over eight hundred bachelors’ granting US educational institutions to connect material taught in higher education to the detailed work activities in the US economy as reported by the US Department of Labor. First, we show how differences in taught skills both within and between college majors correspond to earnings differences of recent graduates. Further, we use the co-occurrence of taught skills across all of academia to predict the skills that will be taught in a major moving forward. Our unified information system connecting workplace skills to the skills taught during higher education can improve the workforce development of high-skilled workers, inform educational programs of future trends, and enable employers to quantify the skills of potential workers.

2. 

   Knowledge Annotation for Intelligent Textbooks

   Mengdi* Wang ,  Hung* Chau ,  Khushboo Thaker , and 2 more authors

   Technology knowledge and learning, 2023

   Abs HTML

   With the increased popularity of electronic textbooks, there is a growing interest in developing a new generation of “intelligent textbooks,” which have the ability to guide readers according to their learning goals and current knowledge. Intelligent textbooks extend regular textbooks by integrating machine-manipulable knowledge, and the most popular type of integrated knowledge is a list of relevant concepts mentioned in the textbooks. With these concepts, multiple intelligent operations, such as content linking, content recommendation, or student modeling, can be performed. However, existing automatic keyphrase extraction methods, even supervised ones, cannot deliver sufficient accuracy to be practically useful in this task. Manual annotation by experts has been demonstrated to be a preferred approach for producing high-quality labeled data for training supervised models. However, most researchers in the education domain still consider the concept annotation process as an ad-hoc activity rather than a carefully executed task, which can result in low-quality annotated data. Using the annotation of concepts for the Introduction to Information Retrieval textbook as a case study, this paper presents a knowledge engineering method to obtain reliable concept annotations. As demonstrated by the data we collected, the inter-annotator agreement gradually increased along with our procedure, and the concept annotations we produced led to better results in document linking and student modeling tasks. The contributions of our work include a validated knowledge engineering procedure, a codebook for technical concept annotation, and a set of concept annotations for the target textbook, which could be used as a gold standard in further intelligent textbook research.

2021

1. 

   Automatic Concept Extraction for Domain and Student Modeling in Adaptive Textbooks

   Hung Chau ,  Igor Labutov ,  Khushboo Thaker , and 2 more authors

   International Journal of Artificial Intelligence in Education, 2021

   Abs HTML

   The increasing popularity of digital textbooks as a new learning media has resulted in a growing interest in developing a new generation of adaptive textbooks that can help readers to learn better through adapting to the readers’ learning goals and the current state of knowledge. These adaptive textbooks are most frequently powered by internal knowledge models, which associate a list of unique domain knowledge concepts with each section of the textbook. With this kind of concept-level knowledge representation, a number of intelligent operations could be performed, which include student modeling, adaptive navigation support, and content recommendation. However, manual indexing of each textbook section with concepts is challenging, time-consuming, and prone to errors. Modern research in the area of natural language processing offers an attractive alternative, called automatic keyphrase extraction. While a range of keyphrase and concept extraction methods have been developed over the last twenty years, few of the known approaches were applied and evaluated in a textbook context. In this paper, we present FACE, a supervised feature-based machine learning method for automatic concept extractions from digital textbooks. This method has been created for building domain and student models that form the core of intelligent textbooks. We evaluated FACE on a newly constructed full-scale dataset by assessing how well it approximates concept annotations produced by human experts and how well it supports the needs of student modeling. The results show that FACE outperforms several state-of-the-art keyphrase extraction methods.

2. 

   Orienting Students to Course Recommendations Using Three Types of Explanation

   Run Yu ,  Zach Pardos ,  Hung Chau , and 1 more author

   In Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization , 2021

   Abs HTML

   An emerging challenge in course recommender systems is explaining to students why they have been recommended particular courses. In the context of a university, it can be valuable for a recommender system to introduce students to courses they may have not otherwise taken but which are still relevant to them. However, there is a tension between these goals as students have less ability to judge the relevance of courses, the less familiar they are with them. In this paper, we explore ways of familiarizing students with recommendations with three types of explanations designed with varying levels of personalization. We conduct a 67 student randomized controlled experiment using two course recommendation engines, content and context-based, and augment recommendations with three different types of explanation. Students rated each course recommended in terms of novelty, unexpectedness, and successfulness (i.e., intent to enroll). We find several statistically significant results, including an increase in serendipity (i.e., unexpectedness + successfulness) when explaining new course recommendations using keywords from courses a student has previously taken.

2020

1. 

   Understanding the Tradeoff between Cost and Quality of Expert Annotations for Keyphrase Extraction

   Hung Chau ,  Saeid Balaneshin ,  Kai Liu , and 1 more author

   In Proceedings of the 14th Linguistic Annotation Workshop , Dec 2020

   Abs HTML

   Generating expert ground truth annotations of documents can be a very expensive process. However, such annotations are essential for training domain-specific keyphrase extraction models, especially when utilizing data-intensive deep learning models in unique domains such as real-estate. Therefore, it is critical to optimize the manual annotation process to maximize the quality of the annotations while minimizing the cost of manual labor. To address this need, we explore multiple annotation strategies including self-review and peer- review as well as various methods of resolving annotator disagreements. We evaluate these annotation strategies with respect to their cost and on the task of learning keyphrase extraction models applied with an experimental dataset in the real-estate domain. The results demonstrate that different annotation strategies should be considered depending on specific metrics such as precision and recall.

2019

1. 

   Data-Assistive Course-to-Course Articulation Using Machine Translation

   Zachary A. Pardos ,  Hung Chau ,  and  Haocheng Zhao

   In Proceedings of the Sixth (2019) ACM Conference on Learning @ Scale , Dec 2019

   Article No.: 22

   Abs HTML

   Higher education at scale, such as in the California public post-secondary system, has promoted upward socioeconomic mobility by supporting student transfer from 2-year community colleges to 4-year degree granting universities. Among the barriers to transfer is earning enough credit at 2-year institutions that qualify for the transfer credit required by 4-year degree programs. Defining which course at one institution will count as credit for an equivalent course at another institution is called course articulation, and it is an intractable task when attempting to manually articulate every set of courses at every institution with one another. In this paper, we present a methodology towards making tractable this process of defining and maintaining articulations by leveraging the information contained within historic enrollment patterns and course catalog descriptions. We provide a proof-of-concept analysis using data from a 4-year and 2-year institution to predict articulation pairs between them, produced from machine translation models and validated by a set of 65 institutionally pre-established course-to-course articulations. Finally, we create a report of proposed articulations for consumption by the institutions and close with a discussion of limitations and the challenges to adoption.

2018

1. 

   Course-Adaptive Content Recommender for Course Authoring

   Hung Chau ,  Jordan Barria-Pineda ,  and  Peter Brusilovsky

   In Proceedings of the 14th European Conference on Technology Enhanced Learning (EC-TEL) , Dec 2018

   Abs HTML

   Developing online courses is a complex and time-consuming process that involves organizing a course into a sequence of topics and allocating the appropriate learning content within each topic. This task is especially difficult in complex domains like programming, due to the incremental nature of programming knowledge, where new topics extensively build upon domain concepts that were introduced in earlier lessons. In this paper, we propose a course-adaptive content-based recommender system that assists course authors and instructors in selecting the most relevant learning material for each course topic. The recommender system adapts to the deep prerequisite structure of the course as envisioned by a specific instructor, while unobtrusively deducing that structure from problem-solving examples that the instructor uses to present course concepts. We assessed the quality of recommendations and examined several aspects of the recommendation process by using three datasets collected from two different courses. While the presented recommender system was built for the domain of introductory programming, our course-adaptive recommendation approach could be used in a variety of other domains.

2. 

   Learning Content Recommender System for Instructors of Programming Courses

   Hung Chau ,  Jordan Barria-Pineda ,  and  Peter Brusilovsky

   In Artificial Intelligence in Education , Dec 2018

   Abs HTML

   In this paper, we present a course-adaptive recommender system that assists instructors of programming courses in selecting the most relevant learning materials. The recommender system deduces the envisioned structure of a specific course using program examples prepared by the course instructor and recommends learning content items adapting to instructor’s intentions. We also present a study that assessed the quality of recommendations using datasets collected from different courses.

2017

1. 

   Content Wizard: Concept-Based Recommender System for Instructors of Programming Courses

   Hung Chau ,  Jordan Barria-Pineda ,  and  Peter Brusilovsky

   In Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization , Dec 2017

   Abs HTML

   Authoring an adaptive educational system is a complex process that involves allocating a large range of educational content within a fixed sequence of units. In this paper, we describe Content Wizard, a concept-based recommender system for recommending learning materials that meet the instructor’s pedagogical goals during the creation of an online programming course. Here, the instructors are asked to provide a set of code examples that jointly reflect the learning goals that are associated with each course unit. The Wizard is built on top of our course-authoring tool, and it helps to decrease the time instructors spend on the task and to maintain the coherence of the sequential structure of the course. It also provides instructors with additional information to identify content that might be not appropriate for the unit they are creating. We conducted an off-line study with data collected from an introductory Java course previously taught at the University of Pittsburgh in order to evaluate both the practicality and effectiveness of the system. We found that the proposed recommendation’s performance is relatively close to the teacher’s expectation in creating a computer-based adaptive course.