Chapter 10 - Institutional Research Support for Accreditation
In 2021, UH Mānoa successfully gained a ten-year reaffirmation during its institutional accreditation review from the Western Association of Schools and Colleges (WASC). The Mānoa Institutional Research Office (MIRO) provided timely data support for the University’s accreditation preparation. Most importantly, its robust data infrastructure helped the University meet the accreditation standards that request institutions make data more accessible, transparent, and consistent for both internal and external users.
Abstract
Introduction
Accreditation is one of the most critical events for any university in the United States. Whether or not a university can successfully be accredited by a federally-recognized accreditor holds many consequences, including impact students’ eligibility for U.S. Federal Student Aid, and students’ ability to transfer credits to other institutions and eligibility for graduate and professional schools, just to name a few.
In November 2021, the University of Hawaiʻi at Mānoa had their intensive accreditation reaffirmation review and successfully earned the maximum 10 years of reaccreditation four months later. Mānoa’s WASC Accreditation Steering Committee has been working since November 2017 to prepare for the 2021 accreditation review. Mānoa Institutional Research Office (MIRO) played a key role in facilitating the university’s reaffirmation efforts. The robust data infrastructure MIRO built that aimed to make data more accessible, transparent, and consistent for both internal and external users, effectively helped the University respond to accreditation standards and other data needs during the University’s accreditation preparation process.
What is Accreditation?
According to the Council for Higher Education Accreditation (CHEA), “Accreditation is a review of the quality of higher education institutions and programs. In the United States, accreditation is a major way that students, families, government officials, and the press know that an institution or program provides a quality education” (“About Accreditation”). Both CHEA and the U.S. Department of Education recognizes four types of accreditors: regional, national faith-related, national career-related, and programmatic accreditors. For a public research university like UH Mānoa, the regional and programmatic accreditors are most relevant, and the university’s accreditation was organized by a regional accreditor.
Figure 1: Four Types of Accreditation
(Corresponding Video Here)
CHEA also stated on their website, “in most other countries, accreditation, or quality assurance is carried out by government organizations. In the U.S, the accreditors are private, non-governmental organizations created for the specific purpose of reviewing higher education institutions and programs for quality” (“About Accreditation”). There are six regional accreditors in the U.S. who can grant or reaffirm four year institutions' accreditation status. UH Mānoa is accredited by the Western Association of Schools and Colleges (WASC). Although institutions can be accredited by any regional accreditors, most choose to work with their respective geographic regions (see Figure 2). As shown on this map, WASC covers institutions in California, Hawai’i, and other institutions in the pacific region.
Figure 2: Map of Regional Accreditors
(Corresponding Video Here)
Addressing Accreditation Standards
For a successful reaffirmation, universities need to address how they meet the WASC standards in their institutional reports. Among the four WASC standards, Standard 4.2 speaks directly to the expectation of institutional research practices (see Figure 3). This standard requires the university to answer specific data inquiries while also efficiently addressing internal and external data needs. It also emphasizes the importance of having data “disseminated internally and externally in a timely manner, and analyzed, interpreted, and incorporated in institutional review, planning, and decision-making” (“Standard 4”). This requirement echoes the mission of the Mānoa Institutional Research Office – to efficiently provide timely information and research-based analysis and make data more accessible to faculty, staff, and the general public.
Figure 3: WASC Standard, Criteria 4.2
(Corresponding Video Here)
An important component for any university’s reaffirmation preparation is to create an institutional report where institutions address required topics by the accreditation agencies based on the findings of the institution’s self-study. Because of how extensive the institutional report is, it requires a lot of data and one of the major data suppliers is the institutional research office. The MIRO team only has 3 full-time staff members, it was challenging to figure out how to address community data needs with such a small staff. Our solution was to create a robust data infrastructure that efficiently processes and disseminates data to internal and external users.
The data infrastructure MIRO built has two components: (1) an internal decision support system that serves our faculty, staff, and administrators and (2) a comprehensive office website that services the external data users. With these two components working together, the MIRO data infrastructure addresses WASC’s request that the Institutional Research Office should provide timely and well-interpreted data to both internal and external users.
Figure 4: MIRO Data Platform Structure
(Corresponding Video Here)
To be more specific, MIRO’s Decision Support System offers Mānoa’s internal users data and analyses in a timely manner. The unique designs and features make data easy to understand for campus decision makers. MIRO also created many help pages and video tutorials to help explain what data means and how to use them.
Figure 5: MIRO Decision Support System for Internal Data Users
(Corresponding Video Here)
Mānoa’s external audiences include prospective students, parents, job applicants, the media, government agencies, and the general public. MIRO’s comprehensive website has rich data resources to serve external users, especially under the Facts tab where they can find a variety of reports. For example, the enrollment data web page provides over a hundred enrollment reports broken down by different demographic groups, colleges and departments, and academic levels (see Figure 6).
Figure 6: Enrollment Reports on MIRO Website
(Corresponding Video Here)
To further help website viewers better understand the data MIRO provides, the office created a series of Analysis Briefs to explain major data trends, published an online Glossary of Terms to introduce specific data terms, and hosted a series of Virtual Symposiums to address key data questions and issues.
Figure 7: Data Support for External Users
(Corresponding Video Here)
Addressing Core Commitments
WASC calls upon institutions to ground their activities in three core commitments and expects institutions to create learning environments that strive for educational excellence and operational effectiveness while serving both students and the public good.
The three core commitments focus on (1) student success, (2) quality and improvement, and (3) institutional integrity, sustainability, and accountability. The Mānoa Institutional Research Office cares about the same things and the staff have contributed to the same goals by providing transparent, consistent, and comprehensive data to UHM campus decision makers.
Figure 8: Data Support for External Users
(Corresponding Video Here)
Student Success Measures
MIRO provides a lot of student success data through our web apps, such as GPA, persistence rate, degree, time to degree, NSSE survey results, and more. The data helps the university track student achievements, evaluate program effectiveness, and holds the institution accountable--thus speaking to all three WASC core components. We won’t elaborate on this important topic in this chapter, but we will go more in depth in the chapter about student success measures.
Figure 9: Student Success Data Provided by MIRO
(Corresponding Video Here)
UH Mānoa’s programs and offices can use MIRO’s web apps to tailor data in ways that best suit their needs, which offers great flexibility to address data inquiries. When preparing for an institutional accreditation review or a programmatic accreditation, both big picture data (either at the institutional or college level) and very specific data are needed. No matter how general or specific it is, reports with historical data and graphs can be generated using MIRO’s web apps within seconds. It can be as general as tracking and displaying the institution’s four-year, six-year, and eight-year graduation rates.
Figure 10: MIRO Graduation Rates Report
(Corresponding Video Here)
Users can also customize the student persistence report to focus on smaller student populations. In this example, the persistence rate graph shows how well full-time female Native Hawaiian students successfully graduate or retain at UH Mānoa after transferring from other UH campuses. This flexibility of the web apps empowers Mānoa campus data users with easy-to-access information and also greatly reduces the institutional research office’s reporting load so MIRO can focus on developing more data visualization tools.
Figure 11: Customizable Persistence Report
(Corresponding Video Here)
NSSE
Like we pointed out in the Chapter about the National Survey of Student Engagement (NSSE), data is a great resource for accreditation preparation. NSSE posted accreditation “toolkits” on their website which maps student engagement results to regional and professional accreditation standards. To fully utilize NSSE data, MIRO designed a set of NSSE data dissemination tools. While Mānoa prepared for the accreditation visit, our IR office has received a lot of positive feedback from different offices, such as the Office of Assessment, Civil Engagement, General Education, and Sustainability. We were told that the NSSE data and MIRO’s NSSE web app tools were extremely helpful for gathering the information they needed.
Figure 12: NSSE Web Apps Developed by MIRO
(Corresponding Video Here)
Special Programs
While the university was preparing for accreditation, MIRO received various requests for comprehensive data analysis to help talk about their work, which came with urgent deadlines. Instead of preparing reports manually, the office utilized homegrown web apps and created a filter called “special programs” and imported student records provided by the corresponding offices. MIRO’s web apps instantly generate customized reports for those programs, enabling staff to retrieve data ready in a smaller fraction of time compared to manual data pulling. If we didn’t create our own data web apps, it would have caused great stress for the IR office to fulfill so many requests before the deadline, which shows how impactful a robust homegrown data system is when addressing urgent data needs.
Figure 13: Addressing Specific Accreditation Data Needs
(Corresponding Video Here)
Qualitative Data
Another important type of data is qualitative data, which could answer questions that numbers cannot. Mānoa’s faculty, staff, and even UHM’s WASC accreditation steering committee have all asked if MIRO has qualitative data on student feedback and satisfaction. To make student feedback available for Mānoa’s decision makers, MIRO made great efforts to develop open-ended question surveys and qualitative data web apps.
One of the most supportive qualitative tools created was the Campus Experience Survey administered in 2016, 2017, and 2018. Each year, MIRO asked students how the university had and/or had not displayed a supportive environment for them.
Figure 14: Open-ended Questions in the Campus Experience Survey
(Corresponding Video Here)
Through those open-ended question surveys, MIRO collected tens of thousands of narrative responses. The results are truly a gold mine of information for the university to dig through and utilize. To make that valuable information accessible to our faculty, staff, and administrators, we developed online data web apps to help users easily navigate through the large amount of random survey results to find what they are looking for.
Figure 15: Qualitative Data Web Apps
(Corresponding Video Here)
While building this theme-indicator-keyword structure, MIRO used the institution’s organizational chart as a reference so that offices and programs could quickly locate student feedback most relevant to them. For example, academic advisors can find out how students answered questions regarding advising by clicking the “advising” tab — over 1,500 responses appear immediately. This valuable data not only helps offices prepare for their sections in the accreditation self-study and institutional report, but can also support their efforts for continuous improvement.
Figure 16: Qualitative Data Web App Design
(Corresponding Video Here)
Another example is the Learning Emporium program, which was mentioned by students time and time again in the college experience survey (see Figure 16). Programs acknowledged and appreciated by students can be considered “best practices” at the university and could be beneficial to include in the accreditation process.
Figure 17: “Emporium” Comments
(Corresponding Video Here)
Ultimately, MIRO’s open-ended question surveys and quantitative data tools are important evidence that UH Mānoa actively seeks to understand and improve student success, operates in a transparent manner, and adapts to changing conditions; all of which are WASC’s core commitments .
Figure 18: “Emporium” Comments
(Corresponding Video Here)
Earth Day Survey
Although the office's plate is always full, MIRO staff try to help with projects they believe are meaningful and critical for the university, especially those with greater causes. When the Sustainability Office reached out for MIRO’s help to design and administer UH Mānoa’s first Earth Day Survey in 2018, a partnership was immediately established. Together, they designed the survey, administered the survey, and created web apps to disseminate both quantitative and qualitative survey data. Because of how successful this survey turned out to be, MIRO decided to administer the Earth Day Survey in 2020 and 2021 and added new questions on how COVID-19 impacted student and employee understandings and concerns regarding sustainability and climate change issues.
When working on this project, MIRO didn’t realize the Earth Day Survey would be such an important source in Mānoa’s institutional report during the accreditation preparation process. The Earth Day Survey results and main findings were included in the Sustainability and Resilience Programs, as well as in the “Courses” section under WASC Standard’s component 8: Academic Innovation & Engaged Learning of Mānoa's Institutional Report
Figure 19: Earth Day Survey Data on UHM’s Institutional Report
(Corresponding Video Here)
Closing Remarks
It is evident that MIRO does not specifically gather and report data for accreditation alone, but has always worked towards supporting student success, institutional effectiveness, and continuous improvement; all of which are WASC’s core commitments as well. Each day, MIRO continues working towards the office mission statement while following its core values.
Figure 20: MIRO Office Mission Statement
(Corresponding Video Here)
The office has grown stronger these past few years while the university was preparing for the accreditation visit, not to merely off the boxes for accreditation but because the staff is constantly seeking ways to improve data quality and accessibility. These daily efforts benefit many major tasks in the university, including accreditation.