Chapter 1 - Develop a Highly Efficient Institutional Research Office
Develop a Highly Efficient
Institutional Research Office
Abstract
While searching for innovative solutions to improve office efficiency and better fulfill the campus’s data needs, the Mānoa Institutional Research Office (MIRO) developed unique insights and strategies to address key bottleneck issues through automating data reporting and reducing repetitive work and inefficient communication.
Introduction
The University of Hawai‘i has ten campuses, comprising six community colleges and three 4-year institutions, which are spread across four major Hawaiian Islands: O‘ahu, Big Island, Maui, and Kaua‘i (see Figure 1). The Mānoa Institutional Research Office (MIRO) provides data support specifically for the University of Hawai‘i at Mānoa (UH Mānoa), which is the largest campus and the only research intensive University in Hawai‘i.
UH Mānoa used to share the same IR office with the greater UH system until MIRO was created around 2007 to better accommodate data needs of Mānoa’s campus. It is a comparatively young office with only three full-time positions. MIRO is responsible for preparing internal and external reports, supporting accreditation and program review data needs, conducting student and employee surveys, and addressing data inquiries from campus decision makers and the general public.
Like many other IR offices, MIRO has limited time, staff, and resources that cannot keep up with the continuously increasing data needs. There is only one way out of this difficult situation: to become highly efficient.
What is Efficiency?
Efficiency at MIRO is defined as how well the office is able to fulfill data reporting responsibilities, how effectively the office can help decision makers access data–especially actionable data–and how successful MIRO efforts are in promoting a data-informed decision-making culture on Mānoa’s campus. Like other IR offices, MIRO has many regular and mandatory reports to prepare, as well as numerous ad-hoc data inquiries to address, which is equivalent to hundreds of smaller tasks each year (see Figure 2).
With her experiences, director Yang Zhang, Ph.D, noticed that one of the most common practices in IR is the “laundry list” approach that processes data requests one at a time. Checking things off a long to-do-list surely offers some immediate satisfaction and creates an illusion of being very productive, but it involves a lot of repetitive work and requires negotiations between IR and other offices for prioritization. Using a “laundry list” approach can result in increasing stress levels and burnout, which is not sustainable nor as productive as one would think. The “laundry list” approach also creates a hierarchical system when supporting data needs. Executive decision makers and higher-level committee requests usually have more impact on the university as a whole, so their data needs often trump others’, leaving some requests on the sidelines.
Because of how time-consuming it is, MIRO has always tried to avoid the laundry list approach. Instead, we have applied a “system-thinking” mindset that allows us to systematically address IR bottleneck issues and provides consistent and standardized data support to a large audience. The 80/20 rule states that roughly 80% of consequences come from 20% of the causes. For those in IR, this could mean that a majority, or 80%, of time is spent conducting repetitive work caused by a small number of bottleneck related issues. If more time is dedicated towards creating efficient solutions to systematically address bottlenecks issues, then it could significantly save time in the long run.
When Dr. Yang Zhang was hired as the MIRO director in 2012, she visited different offices and departments to identify their data needs. She realized that the biggest data challenge at our university was about data accessibility caused by a few major bottleneck issues. So, Yang rewrote MIRO’s mission statement and created a plan to systematically address those bottleneck issues. She then shared the plan with her supervisor and other colleagues to seek their understanding and support.
Creating systematic changes required a lot of foundational work at the beginning, but was incredibly rewarding. Within the first year alone, MIRO completed the Common Data Set two months earlier than usual, cut the external survey preparation time by 75%, and shortened the preparation time of some data requests from hours to seconds. Once MIRO achieved some momentum, the time the office saved was then used to address more bottleneck issues and cut more repetitive work, moving in an upwards spiral towards higher and higher efficiency.
Data Consistency
The first IR bottleneck issue is about unclear and inconsistent data methods. This occurs when data definition and calculation methods are not clearly defined nor well documented, and is perpetuated when there are personnel changes in the office. This is not a unique situation for MIRO; many IR professionals have run into this challenge where they have to frequently guess data definitions and attempt to match the numbers reported in the past with the present. At MIRO, this issue is addressed by clarifying data definitions and calculation methods whenever possible. We developed an online glossary of terms, or rather a data dictionary, using both industry and local data knowledge for our campus data users.
IPEDS and CDS are main sources of industry data knowledge that tell IR how the higher education system defines and calculates data. The local data knowledge includes two parts: (1) how data is defined and structured in the university’s database, and (2) how data is collected and processed by different offices. When readily used calculation methods are not available, MIRO creates new methods by collaborating with data users and colleagues who have local data knowledge. These colleagues help to review the newly created calculations and give MIRO feedback on whether the calculation methods are appropriate and whether the data generated is useful.
After finalizing the data definition and calculation methods, MIRO must then organize and document them well so they’re easy to locate when needed. To document data definitions and calculation methods on MIRO’s website and internal Decision Support System, MIRO uses digital analysis briefs, video tutorials, help pages, and a glossary of terms. To document software installation, programming code, and data queries, MIRO mainly uses Confluence and Bitbucket that allow teams to edit shared pages and collaborate on code.
All this work takes a tremendous amount of time and patience, but the process ultimately saves time and frustration in the long run. With clearer definitions and organized documentation, institutional researchers can free themselves from investigating how data was reported in the past and they can greatly reduce data discrepancies caused by inconsistent calculation methods.
Data Extraction & Preparation
The second IR bottleneck issue is about data extraction and preparation. When data is not readily prepared for reporting, a copious amount of time is spent on making data ready before IR can conduct any analysis. Most data requests are quite similar, so a great amount of time could be exhausted in repetitive work.
MIRO first took an IR approach to address this bottleneck issue. Our IR analyst carefully examined data downloaded from the university’s database and wrote SPSS syntax to transform data to meet our reporting needs. Thanks to her efforts, some of the non-useful data variables were deleted, leaving room to create new variables that could be readily used for reporting. This IR database transformation helped immensely that MIRO no longer needed to prepare data if they were already transformed and ready for reporting.
Traditional IR knowledge and skills, however, have great limitations in further addressing data extraction and preparation needs. Traditional IR expertise focuses on statistical analysis and usually uses software like SPSS, SAS, Access, Excel, or R. Although these are powerful data analysis tools, they are not the most efficient ways to extract and prepare data. To be more efficient, MIRO went beyond the typical IR analysis skills to include IT programming and computing as core skill sets in our office.
In 2013, MIRO made the most important decision to change one of the two IR analyst positions to an in-house, IT specialist position. It ultimately allowed us to create an “easy button'' to address complicated data issues with instant solutions.
Programming has been a significant game changer for the office. Through automating a majority of the work, programming turns some of MIRO's wildest imaginations into innovations, and enables the office to "scale up" IR production without spending as much time and effort. The benefits of having an in-house IT specialist are tremendous. Different from working with programmers at an IT office, having an in-house IT specialist prioritizes IR’s needs while gaining a great deal of IR data knowledge, which can help them advance IR’s work to the best of their abilities. Over the years, MIRO kept exploring how to merge the two very different skills of IR and IT. MIRO’s director continued to encourage and support IR and IT’s efforts in expanding their knowledge to each other’s fields, both of them also learned a lot from each other while collaborating.
IR and IT Collaboration from IR’s Perspective
MIRO’s IR analyst used to use a variety of software at different stages of the data preparation process such as extraction, conversion, analysis, and reporting. Extra time and unexpected errors were more likely to occur when transforming data between multiple software programs. When those errors happened, it would take hours or even days to detect and fix the problem, which took time away from working on other projects and caused personal stress.
After witnessing how quickly the newly hired IT specialist used Structured Query Language (SQL) to extract and prepare data, the IR analyst was inspired to learn programming as well. She started with learning about a simple SQL query with the help from the IT specialist and relied on free online SQL resources for questions. Later, she took online courses from Udemy and enrolled in a computer science class at UH-Mānoa to learn programming systematically.
With improved understanding about SQL, the analyst began writing SQL queries for data extraction and analysis. When she saw an improvement in work efficiency after applying newly learned knowledge in daily work, it kept her motivated to continue advancing her knowledge and application of programming. Soon she was able to write queries to address data requests, validate data prepared by the IT specialist, and prepare the CDS and external surveys with ease. The IR analyst was able to address many issues through SQL queries alone and didn’t have to rely on different software programs throughout the data extraction, conversion, and analysis processes. This transformation saves time, reduces the risk of errors, and greatly improves MIRO’s work efficiency. Another major benefit of learning programming is the ability to collaborate and communicate better with IT colleagues. Using the programming knowledge acquired, IR analysts can explain the data requests more efficiently while speaking more confidently and effectively about data issues noticed. Even basic programming skills can significantly improve IR’s work efficiency and broaden the scope of tasks that IR can do. With the time saved from cutting a lot of daunting and repetitive work, it frees IR’s time to focus on new learnings to further improve work efficiency and quality.
IR and IT Collaboration from IT’s Perspective
Responsibilities for an IT specialist include pulling data and updating MIRO’s databases every semester, as well as writing SQL queries when developing MIRO’s Decision Support web apps. Both parts require knowing the data well in order to recognize possible data errors and accurately address different data needs.
Universities naturally track an enormous amount of data, and learning about it all is a never-ending task. The sheer amount of data available poses some challenges when extracting data from the university’s official databases, as well as when cleaning and transforming data to better suit MIRO’s specific reporting needs. Much of the code used to do this data processing was written by MIRO’s previous IT specialists and is still used since it still serves us well today. But over the years, the office has faced new challenges stemming from the increasing size and scope of our database.
While MIRO wants to make more data available, each new variable added brings more complexity to the database structure and the data transformation process, adding more potential for errors to be introduced. Understanding how the data is structured, how the different variables relate to each other and to other data sets, and how the data might be interpreted by people who aren’t familiar with it are all key areas of IR that are usually associated with a data analyst. Sharing some of these crossover skills in programming and data analysis enables both MIRO’s IT and IR analysts to work more efficiently without having to constantly pass simple tasks back and forth. Instead, they can focus on more complex data questions.
Regarding the technical aspects of those web apps, MIRO uses PHP scripting language for most of the app development to handle back-end processing of the data. The foundation for our web apps was also built by previous IT specialists, which really help streamline the process of building new web apps. Now it’s quite easy to develop a new app by re-using code that’s already been written for the existing ones.
Updating our existing web apps is also straightforward: much of the code can be easily tweaked to accommodate new data points or filtering variables. By making it faster and easier to do the small changes, it frees up time to work on new apps to systematically address more data needs and other initiatives that MIRO undertakes. Aside from the office’s own team, MIRO relies on UH-Mānoa faculty & staff for feedback regarding our reports and tools. Their input helps MIRO make better decisions on how to better serve the campus as an IR office.
Data Reporting & Dissemination
Both IR and IT specialists bring up the importance of data visualization and report automation, which relates to the third IR bottleneck issue about data reporting and dissemination. Preparing a report takes time, especially the formatting part. It is tedious, repetitive, and not the best way to utilize IR’s skills. A lot of IR offices use data visualization software like tableau and PowerBI to address this bottleneck issue; why would MIRO take a different approach to create personalized own software?
While exploring data visualization solutions, MIRO did not have the funding to purchase a Tableau license, but there was a vacant IR position that could be repurposed. This was the practical reasoning behind MIRO’s decision, but it wasn't the main reason.
Software like Tableau is not meant to conduct sophisticated backend data preparation work and does not have the same level of freedom and flexibility in designing features of data reports. Having an in-house IT specialist, however, allows the office to do both sophisticated backend and frontend work.
The data visualization web apps MIRO created in-house are available through the Decision Support System, which is an internal, centralized data platform. This system was built to give campus data users access to data anytime and anywhere, and to address commonly asked data questions. The most powerful feature of MIRO’s web apps is the ability to customize reports using dozens of filters that draw data from the university’s database. Data users often wonder what data is available, so MIRO’s web apps were designed to allow users to see all possible data to use. Generically prepared reports, no matter how sophisticatedly designed, are not able to accommodate different needs from a large number of users. So when MIRO designed their own data tools, the office wanted to offer the most comprehensive data access and most flexibility in design that would allow users to customize reports based on their own needs.
We learned from a recent web app user feedback survey that MIRO web app users thought the variety of web app filters was the most helpful feature, followed by the design of the report and the report download functionality. In other words, MIRO’s web apps give data users better access to data, more flexibility in customizing reports, and easier downloading options. Another key takeaway from the survey was that the web apps were used to serve many different purposes such as program review, assessment, accreditation, program management, grant application, new program proposals, faculty hiring justifications, and more.
Some users said they found the web apps easy to use and appreciated how quickly they could use the apps to address urgent data needs. One respondent said they especially appreciated the flexibility of using filters to break down data, and another found the student survey and qualitative data web apps extremely helpful. Figure 14 also shares other users’ feedback found from MIRO’s “Web App User Survey”.
The web apps also help MIRO’s own work efficiency, frequently utilizing the tools to generate a large number of reports published on our office website. For example, using the enrollment web app can generate a hundred enrollment reports within an hour, which could take days if done manually. Web apps are also used to quickly address a large amount of data inquiries received. Having an in-house IT specialist makes it possible to quickly add a new filter or create a new web app tool.
Data Communication
Making programming an essential skill set helped MIRO address both data extraction and data reporting bottleneck issues, but it led to another major bottleneck issue: communication. Communication bottleneck issues often occur when IR has to go back and forth between other offices in order to collect information from them. These issues also happen when IR needs to clarify colleagues’ data requests and when recipients need help understanding the reports created. Emailing other offices about repeated information could cost IR massive amounts of time, which is not the best use of time and causes distractions for both IR and other offices’ personnel.
Preparing for the Common Data Set and external surveys like U.S. News, Peterson’s, and the College Board Survey is one of MIRO’s major responsibilities. While the office does take care of most data calculations, some information can only be collected from other offices. Since IR needs to coordinate multiple external survey submissions throughout the year, the laundry list approach will only generate massive amounts of email threads and cause distractions for IR and other offices. So, MIRO took a systematic approach and consolidated CDS and all the external survey questions into a master document, then used standardized email templates and tools to collect information from multiple offices at once. Thanks to this streamlined process, it eliminates a majority of manual work and excessive individual communication threads. It also lowers the risk of human errors and helps both IR and other offices stay organized and less distracted.
MIRO also runs into this communication bottleneck issue when needing to clarify what Mānoa data users need before preparing the data and knowing how to help them understand the data after it has been prepared. To avoid repeated explanation of the same information, MIRO created different resources to help answer most-commonly asked questions. The online glossary of terms, web app help pages, analysis briefs, and video tutorials not only provide instant help when users have questions, but it also helps save IR’s time. Now, whenever someone has a question, answers can easily be sent without having to draft them from scratch.
Ultimately, systematically addressing data methods like data extracting, data reporting, and data communication bottleneck issues allows IR to significantly reduce, or even eliminate, repetitive work. Better data systems also give IR staff more convenience, consistency, and efficiency in their work. Saving the office from the tedious repetitive work also made it possible to learn new skills; now more effort can be put into finding creative solutions to better serve the campus.
Essential Skills for an Office
Some skill sets and talents MIRO considers essential are not always common across the board in institutional research. The core skills for Mānoa’s office are data management and analysis, which are essential skills for all IR offices in general. Institutional researchers need to know how to use research and analysis methods to prepare reports and how to address data questions. Different IR offices may use different data management and analysis tools, but MIRO mainly uses SurveyMonkey and Google folders for data collection. For data analysis, the office often uses SPSS, Excel, and SQL.
The second critical skill is programming. With a small IR office of three full-time staff members, we always make sure to have an in-house IT position. MIRO’s IT specialist created a dynamic and powerful Decision Support System and ensured quick turnaround time for many sophisticated data requests. From a technical standpoint, MIRO uses SQL and PHP to handle the back-end data processing we mentioned earlier and to build data web apps. The office also utilizes free resources like Highcharts to create data visualizations and Bitbucket to host website codes and other SQL scripts.
The third type of important skills are multimedia and performing arts, which encompass a wide variety of skills including (but not limited to) script writing, graphic design, video and audio editing, website editing, and script narrating. Data is not the easiest topic to engage audiences with, which is why graphic designers are needed to help turn ideas into easily digestible and visually appealing content. Performing artists also help engage with our audiences in video presentations and live events, as they can express their ideas and feelings effectively through performance. In terms of tools, MIRO has used Wordpress, Indesign, Canva, Camtasia, iMovie, Zoom; and other designing, editing, and recording software based on staff members’ preferences.
The fourth set of skills that MIRO values are marketing and campaigning to help spread our services and ideas to internal data users and external collaborators. Although MIRO has done many NSSE survey administration campaigns, campaigning about our office’s services is a new territory that Mānoa’s office just started exploring these past few years. MIRO uses Mailchimp to build the office’s audience group by sending out monthly newsletters. MIRO also conducts virtual symposiums using zoom webinar and broadcasts the symposiums through recorded videos and Apple Podcasts.
Closing Remarks
The core skills MIRO found necessary when building a highly efficient IR office continue to be essential in the office today. How offices decide to spend their time impacts the quality of their work-life balance. MIRO did not want to waste time doing repetitive work, so we took the matter into our own hands by creating “easy buttons” that do higher quality work in more efficient manners. In turn, the saved time was used to focus on more meaningful and interesting projects. Every office can have proactive communicators, forward thinkers, and courageous leaders; it's up to each IR office to identify the core changes they want to make within their offices and need to center their efforts and desired skill sets around that.
This chapter is mainly geared towards institutional research professionals; however, repetitive work and team efficiency are not unique issues for IR. We hope this can inspire non-IR colleagues to reassess their work routines and find opportunities to be more efficient.