program evaluation higher education

Search form

• About Faculty Development and Support
• Programs and Funding Opportunities

Consultations, Observations, and Services

• Strategic Resources & Digital Publications
• Canvas @ Yale Support
• Learning Environments @ Yale
• Teaching Workshops
• Teaching Consultations and Classroom Observations
• Teaching Programs

Spring Teaching Forum

• Written and Oral Communication Workshops and Panels
• Writing Resources & Tutorials
• About the Graduate Writing Laboratory
• Writing and Public Speaking Consultations
• Writing Workshops and Panels
• Writing Peer-Review Groups
• Writing Retreats and All Writes
• Online Writing Resources for Graduate Students
• About Teaching Development for Graduate and Professional School Students
• Teaching Programs and Grants
• Teaching Forums
• Resources for Graduate Student Teachers
• About Undergraduate Writing and Tutoring
• Academic Strategies Program
• The Writing Center
• STEM Tutoring & Programs
• Humanities & Social Sciences
• Center for Language Study
• Online Course Catalog
• Antiracist Pedagogy
• NECQL 2019: NorthEast Consortium for Quantitative Literacy XXII Meeting
• STEMinar Series
• Teaching in Context: Troubling Times
• Helmsley Postdoctoral Teaching Scholars
• Pedagogical Partners
• Instructional Materials
• Evaluation & Research
• STEM Education Job Opportunities
• AI Guidance for Faculty and Students
• Yale Connect
• Online Education Legal Statements

You are here

Program assessment.

Program evaluation looks at the parameters, needs, components, and outcomes of program design with an eye towards improving student learning. It involves a complex approach, taking into consideration needs assessment, curriculum mapping, and various models of program review.

Needs Assessment

Curriculum mapping, program review, kirkpatrick model, data collection, you may be interested in.

Reserve a Room

The Poorvu Center for Teaching and Learning partners with departments and groups on-campus throughout the year to share its space. Please review the reservation form and submit a request.

Hosted annually, the Spring Teaching Forum fosters discussions about teaching and learning within the Yale community.

The Poorvu Center for Teaching and Learning routinely supports members of the Yale community with individual instructional consultations and classroom observations.

•   [email protected]

Higher Education Administration (HEA) Guide

A review of quantitative and qualitative analysis.

Need a refresher on Quantitative and Qualitative Analysis? Click below to get a review of both research methodologies.

• Quantitative and Qualitative Analysis This link opens in a new window

Program Evaluation and Planning

Image by Kelly Sikkema, retrieved via Unsplash

From data analysis to program management methods and more, evaluating and planning for the success of each program is a crucial aspect of Higher Education Administration. Below you will find some useful articles and reports to help bring context to this important element of higher education leadership. 

Useful Articles

Below you will find a sample of reports, case studies and articles that outline the process of program evaluation, planning and analysis. Click through and read on for more information. 

• The Feasibility of Program-Level Accountability in Higher Education: Guidance for Policymakers. Research Report Policymakers have expressed increased interest in program-level higher education accountability measures as a supplement to, or in place of, institution-level metrics. But it is unclear what these measures should look like. In this report, the authors assess the ways program-level data could be developed to facilitate federal accountability.
• Improving Institutional Evaluation Methods: Comparing Three Evaluations Using PSM, Exact and Coarsened Exact Matching Policymakers and institutional leaders in higher education too often make decisions based on descriptive data analyses or even anecdote when better analysis options could produce more nuanced and more valuable results. Employing the setting of higher education program evaluation at a midwestern regional public university, for this study we compared analysis approaches using basic descriptive analyses, regression, standard propensity score matching (PSM), and a mixture of PSM with continuous variables, coarsened exact matching, and exact matching on categorical variables. We used three examples of program evaluations: a freshman seminar, an upper division general education program intended to improve cultural awareness and respect for diverse groups, and multiple living learning communities. We describe how these evaluations were conducted, compare the different results for each type of method employed, and discuss the strengths and weaknesses of each in the context of program evaluation.
• Data-Informed Policy Innovations in Tennessee: Effective Use of State Data Systems This link opens in a new Analysis of student-level data to inform policy and promote student success is a core function of executive higher education agencies. Postsecondary data systems have expanded their collection of data elements for use by policymakers, institutional staff and the general public. State coordinating and governing boards use these data systems for strategic planning, to allocate funding, establish performance metrics, evaluate academic programs, and inform students and their families. This report discusses efforts at the Tennessee Higher Education Commission to support policy innovation with data and information resources.

Other Resources

• << Previous: Contemporary Issues In Higher Education
• Next: Legal and Regulatory Research >>

Solutions : Program Evaluation

Delivering a high-quality educational program requires measuring its effectiveness and making necessary adjustments along the way.

At the Johns Hopkins School of Education, we’re committed to improving educational programs and initiatives by expanding the body of evidence on what makes them effective.

From assessing what your stakeholders really think about your school community to identifying at-risk students early on, our expertise leads to informed decisions.

Our experts evaluate programs for all grade levels and content areas, as well as overarching topics in education, including school completion, post-secondary readiness (career and college), school reforms and partnerships, education technology, education policy, and community engagement.

Discover Effective Program Evaluation Solutions

Ela knowledge map™.

PROGRAM EVALUATION

An in-depth proprietary analysis that maps every piece of student-facing material across an ELA curriculum.

• School districts
• School systems
• Education leaders

Social Studies Knowledge Map™

A tool that analyzes a social studies curriculum by mapping knowledge domains and the texts that are studied.

• State, district, and school leaders

School Culture 360™ Survey

A survey that produces an “under the hood” analysis of behaviors and beliefs that influence student success.

• K-12 school communities

Teacher Survey of Curriculum Use

A full picture of teachers’ use of curriculum materials and their satisfaction level with those materials.

Policy and Implementation Guidance

The Johns Hopkins Institute for Education Policy strives to ensure that policy and practice are guided by research.

Online Referral Tool

A technology solution that promotes early identification and referral for infants and toddlers who may be eligible for early intervention services.

On Track 4 Success Program

A program that places early warning response systems in schools in under-resourced central Maryland neighborhoods.

On Track to Career Success

Enabling all students, including those facing the toughest challenges, to build educational, social-emotional, and workplace skills that lead to success.

Case Study: Fairfax County Public School System Rethinks Student Success

One of the largest public school districts in the United States, Fairfax County Public Schools is rethinking what student success means in a technology-driven society.

FCPS partnered with the Center for Research and Reform in Education to evaluate a phased rollout of its FCPSOn program over several years. An innovative approach to technology-enhanced education and blended learning, FCPSOn involves the distribution of personal laptops for students, targeted professional development for teachers, unique tools for technology-enhanced classrooms, and moving beyond high-stakes assessment.

Focus areas examined in the first three years included student technology use, teacher technology practices, and family technology perceptions. Among other key points, CRRE found high agreement among teachers  — over 75% — that students had improved in their use of technology as a learning tool each year.

More Program Evaluation Solutions from the Johns Hopkins School of Education

City year network school improvement.

Continuous improvement practices that help students complete eighth grade on track to high school graduation.

Implementation Evaluation

Qualitative case studies that generate formative information.

Efficacy Evaluation

Smaller-scale, formative studies that focus on how a program operates and affects educational outcomes in pilot schools.

Effectiveness Evaluation

Larger-scale, summative studies that focus on the success of a program in improving outcomes in rigorous, nonrandomized (“quasi”) experimental studies (QEDs), or randomized controlled trials (RCTs).

Building an Evidence Portfolio

Personalized goals and plans for obtaining evidence to support your products and services.

Instructional Design and Curriculum Review

An expert review of teacher and student materials, including the written, taught, and learned curriculum, conducted by the Center for Research and Reform in Education.

Our Program Evaluation Experts

The Johns Hopkins School of Education is home to a diverse group of scholars, researchers, analysts, and administrators with expertise in educational program evaluation.

David Steiner, PhD

Professor Executive Director, Johns Hopkins Institute for Education Policy

Affiliation

Institute for Education Policy, SOE Leadership

Curriculum Studies, Education Policy & Politics, Research, Evaluation & Assessment

Ashley Rogers Berner, PhD

Associate Professor Director, Johns Hopkins Institute for Education Policy

Jennifer Morrison, PhD

Associate Research Professor Deputy Director of Evaluation Research, Center for Research and Reform in Education

Center for Research and Reform in Education

Curriculum Studies, Educational Technology, Learning & Instruction, Research, Evaluation & Assessment

Steven M. Ross, PhD

Professor Director, Center for Research and Reform in Education

Center for Research and Reform in Education, SOE Leadership

Curriculum Studies, Research, Evaluation & Assessment

Certificate in Education Program Evaluation

Develop expertise in program evaluation theory, methods, and skills to effectively measure and report on the success of programming and add value to your organization.

Curriculum & Schedule

How to register, tuition & funding.

The Certificate in Education Program Evaluation prepares you with an advanced understanding of program evaluation theory, methods, and applications for the 21st century. Through case studies and hands-on exercises, you’ll develop the well-rounded skills and expertise needed to support and influence programs across not only the education sector, but also non-profit organizations, government, and associations.

In the classroom, you’ll learn from academics and advanced practitioners as you work toward designing and presenting your own program evaluation. Upon completing the program, you’ll be able to effectively measure, evaluate, and report on the success of programming within your organization.

Ideal for Professionals in education, non-profits, the public or private sector

Duration 7 weeks

Tuition $2,797

Format Online (SDL+)

Schedule Weekly Zoom class session

Semester of Entry Fall, spring

Upon successful completion of the certificate, you‘ll be able to:

• Compare evaluation theories and techniques
• Identify design structure of an evaluation tool
• Apply appropriate research methodology to program evaluations
• Design a program or policy evaluation outline
• Leverage evaluation findings to influence future change

Testimonials from current students and alumni.

Amy did a really exceptional job keeping the conversations within scope of relevance. The activities also helped me develop the mindset to critically think about programs I develop and how I would eventually want to evaluate them.

You must successfully complete all three modules for a total of 4.9 Continuing Education Units (CEUs), which is equivalent to 49 contact hours.See detailed description of the modules.

• Program Planning, Analysis and Evaluation (Required; 1.4 CEUs)
• Research Methods (Required; 2.1 CEUs)
• Program Evaluation Design (Required; 1.4 CEUs)

What is Self-Directed Learning Plus? Self-Directed Learning Plus (SDL+) courses are learning experiences designed by Georgetown University instructional designers to maximize flexibility for your weekly learning schedule, while also providing opportunities for real-time engagement and networking with our faculty and other professional learners.

There is a required Zoom class session each week. Professional learners who cannot attend the class session will have the opportunity to satisfy the course requirement by making arrangements with the instructor, watching the recorded video, or responding to discussion questions. In order to maintain flexibility, the dates/time of the weekly Zoom lesson will be confirmed at the start of the course and will depend upon the location of the professional learners in that cohort.

Each course module will have assignment and assessment deadlines to support students in progressing in a timely manner to the successful completion of the certificate

Course Schedule

Please review the refund policies in our Student Handbook before completing your registration.

Degree Requirement

You must hold a bachelor's degree or the equivalent in order to enroll in our certificate programs.

Registration

This certificate is an open-enrollment program. No application is required. Click the "Register Now" button, select your courses, and then click "Add to Cart". Course registration is complete when your payment is processed. You will receive a confirmation email when your payment is received. Please retain the payment confirmation message for your records.

You can combine on-campus and online courses (if available) to complete your certificate. Depending on the certificate program, we may suggest taking courses in a specific order, but this is not a requirement.

Most students register for all courses at the same time and complete their certificate within a few months. However, you may choose to register for courses one by one over time. Once you begin a certificate, you have up to two years from the time you start your first course to complete all required courses.

International Students

International students who enter the U.S. on a valid visa are eligible to enroll in certificate courses. However, Georgetown University cannot sponsor student visas for noncredit professional certificate programs.

A TOEFL examination is not required for non-native English speakers but students are expected to read, write, and comprehend English at the graduate level to fully participate in and gain from the program.

Students from most countries may register for our online certificate programs; however, due to international laws, residents of certain countries are prohibited from registering.

Total program tuition for all 4.9CEUs (7 courses) is $2,797. All course materials are included.

Noncredit professional certificates do not qualify for federal financial aid, scholarships, grants, or needs-based aid. However, several finance and funding options do exist, as listed below.

Some employers offer funding for employee education or professional development. If an employer guarantees payment for employee education and training, Georgetown will accept an Intent to Pay form . If you are using employer sponsorship or training authorizations, you must submit an Intent to Pay form with your registration.

If your employer will pay for your tuition, select “Third-Party Billing” as your method of payment when you register for courses online. Please submit an Intent to Pay form indicating that your employer or another third party should be billed for tuition. Invoices will not be generated without this form on file.

• Pay training and education expenses from appropriated funds or other available funds for training needed to support program functions
• Reimburse employees for all or part of the costs of training or education
• Share training and education costs with employees
• Pay travel expenses for employees assigned to training
• Adjust an employee's normal work schedule for educational purposes not related to official duties

Georgetown accepts Standard Form-182 (SF-182) for training authorizations from the federal government.

*Federal employees should ask the appropriate budget officer about training budgets available.

Eligible Georgetown employees may use their Tuition Assistance Program (TAP) benefits to fund 90% of the certificate program tuition—employees will be invoiced for the remaining 10% of tuition and must pay any other charges associated with their certificate program. Employees using TAP benefits may work directly with the HR Benefits Office to ensure payment prior to the start of any course. This payment option is only valid if registration occurs at least 10–14 business days prior to the start date of the first course. Any fees incurred due to course withdrawal are the student’s responsibility and are not funded by Georgetown University TAP. For questions regarding TAP benefits, please contact the HR/Benefits Office at [email protected] or (202) 687-2500.

SCS is registered with GoArmyEd.com to accept SF-182 training authorization forms. GoArmyEd.com is the virtual gateway for all eligible active duty, National Guard, and Army Reserve soldiers to request Tuition Assistance (TA) online. GoArmyEd.com is also the virtual gateway for Army Civilians to apply for their Civilian education, training, and leadership development events.

The professional certificate programs offer an interest-free payment plan for certificate programs that are more than one month in duration and for which the total tuition is greater than or equal to $4,000. The payment plan is structured in the following manner:

• Payment #1: A down payment of 25% of the total tuition balance must be paid online (within 72 hours after you register and select Payment Plan) via the Noncredit Student Portal . Please submit your down payment as soon as possible.
• Payments #2, #3, and #4: Your remaining balance will be due in three (3) equal monthly installments beginning 30 calendar days after your down payment is processed. Your monthly payments must be paid via credit card in the Noncredit Student Portal . You will be able to access each invoice and payment due date in your student account.

PLEASE NOTE: Automatic Payment Service is not available. You must make each subsequent payment via the Noncredit Student Portal .

A number of tuition benefits are available through the Department of Veterans Affairs and under various parts of the GI Bill ® . Please visit the Resources for Military Students page for additional information and instructions.

Some students choose to finance certificate programs with private education loans. Students are responsible for contacting lenders directly to find out if a noncredit professional certificate program is eligible for a loan. While Georgetown University will not recommend specific lenders, it will certify loans for eligible programs from approved lenders.

For eligible noncredit professional certificate programs, Georgetown University will certify loan amounts up to the full cost of tuition for the program. Tuition does not cover books, travel, or living expenses. Please see individual program pages for tuition rates.

Georgetown University has a unique campus code for Sallie Mae. Our Sallie Mae branch code is 001445-99.

You must be approved for a loan before registering for courses. Follow these steps to pursue a loan option:

• Check the list of lenders that have offered private education loans in the past to Georgetown University students.
• Contact the lender and confirm your program is eligible for a private education loan.
• Obtain the necessary paperwork and apply for the loan.
• Georgetown will certify loan amounts based on the information below. Please note that our branch code is 001445-99.
• Payment sent to Georgetown: Select “Third-Party Payment” at the time of registration if the lender is sending funds directly to Georgetown.
• Enter the information about the lender and then contact Noncredit Student Accounts at [email protected] .

Note: It is your responsibility to contact Georgetown University Noncredit Student Accounts at [email protected] to ensure that your loan is processed.

While you may choose to complete your certificate program in one semester, many programs (but not all) allow up to two years to complete all requirements. As a result, you may choose to register for required and elective courses over several semesters to spread out the cost of tuition over time. We generally offer every course in a program each semester, so you'll have many opportunities to enroll in required and elective courses within the two-year time frame.

Tuition Discounts

Only one tuition discount may be applied at the time of registration. Tuition discounts cannot be combined. Tuition discounts are not applied retroactively. Please contact [email protected] with any questions.

Georgetown University alumni and SCS certificate completers are eligible to receive a 30% tuition discount for many certificates offered within SCS’s Professional Development & Certificates (PDC) portfolio. When registering for an eligible certificate through the SCS website, you will see the "30% Georgetown Alumni Discount" as an option. The Enrollment Team will then verify your eligibility status as a Georgetown University alumnus or certificate completer.

Georgetown SCS offers a 20% discount for eligible certificates to organizations that register 5 or more employees for the same certificate cohort at the same time. Eligible organizations include government agencies, nonprofit agencies, and for-profit businesses. Please contact [email protected] for steps and procedures to ensure your group has access to the discount.

Employees of Boeing receive a 10% tuition discount on select programs and courses

Employees of companies that belong to the EdAssist education network may receive a 10% tuition discount on select programs and courses. Contact EdAssist directly to find out if you qualify.

Eligible federal employees across the country receive a 10% scholarship applied to the current tuition rate for all SCS degree programs and professional certificate programs each academic semester. Please contact [email protected] for steps to be added to this discount group.

Professional Certificates Help You Retool, Learn New Skills

In an age of accountability, program evaluation has become a key skill, ‘flex learning’ recreates classroom experience online.

Still Have Questions?

Certificate Admissions and Enrollment Email: [email protected] Phone: (202) 687-7000

Student Accounts Email: [email protected] Phone: (202) 687-7696

Certifying Military Benefits Email: [email protected] Phone: (202) 784-7321

Want to learn more?

Request information to find out the latest on our programs.

All fields are required.

• Summer 2024
• Spring 2025
• Summer 2025

* indicates required field

Choose Your Term

This program has multiple applications available. Please select your preferred term.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• AEM Educ Train
• v.6(Suppl 1); 2022 Jun

Program evaluation: An educator's portal into academic scholarship

Shera hosseini.

1 Faculty of Health Sciences, McMaster Institute for Research on Aging, McMaster Education Research, Innovation, and Theory, McMaster University, Hamilton Ontario, Canada

Yusuf Yilmaz

2 McMaster Education Research, Innovation and Theory (MERIT) Program & Office of Continuing Professional Development, Faculty of Health Sciences, McMaster University, Hamilton Ontario, Canada

3 Department of Medical Education, Faculty of Medicine, Ege University, Izmir Turkey

Kaushal Shah

4 Department of Emergency Medicine, Weill Cornell Medical School, New York New York, USA

Michael Gottlieb

5 Department of Emergency Medicine, Rush University Medical Center, Chicago Illinois, USA

Christine R. Stehman

6 Department of Emergency Medicine, University of Illinois College of Medicine ‐ Peoria/OSF Healthcare, Peoria Illinois, USA

Andrew K. Hall

7 Department of Emergency Medicine, University of Ottawa, Ottawa Ontario, Canada

8 Royal College of Physicians and Surgeons of Canada, Ottawa Ontario, Canada

Teresa M. Chan

9 Faculty of Health Sciences, McMaster University, Hamilton Ontario, Canada

10 Division of Emergency Medicine, Department of Medicine, McMaster University, Hamilton Ontario, Canada

11 McMaster Program for Education Research, Innovation, and Theory (MERIT), McMaster University, Hamilton Ontario, Canada

12 Department of Health Research Methodology, Impact, and Evidence, McMaster University, Hamilton Ontario, Canada

Program evaluation is an “essential responsibility” but is often not seen as a scholarly pursuit. While Boyer expanded what qualifies as educational scholarship, many still need to engage in processes that are rigorous and of a requisite academic standard to be labelled as scholarly. Many medical educators may feel that scholarly program evaluation is a daunting task due to the competing interests of curricular change, remediation, and clinical care. This paper explores how educators can take their questions around outcomes and efficacy of our programs and efficiently engage in education scholarship. The authors outline how educators can examine whether training programs have a desired impact and outcomes, and then how they might leverage this process into education scholarship.

INTRODUCTION

Program evaluation has been referred to as an “essential responsibility” for those tasked with the oversight of medical training programs, 1 but it is striking how little of this program evaluation work is labelled as scholarly, and how rarely this work translates into academic scholarship. While what qualifies as educational scholarship has been expanded well beyond traditional peer‐reviewed publications to include the scholarship of teaching, discovery, integration, and application, 2 there is still a need to engage in processes that are rigorous and of a requisite academic standard to be labeled as scholarly. 3 However, being asked to both create educational deliverables and innovate within this context is often already above and beyond the duties of overworked and under‐supported medical educators. Many medical educators may feel that scholarly program evaluation is a step too far—with so many competing interests, it can be difficult to find the “bandwidth” to accomplish these scholarly tasks. 4 Don’t we all wonder about the outcomes and efficacy of our programs? Were our programs received as they were intended? And finally, is my training program having the desired impact and outcomes? And if so, wouldn’t it be nice to generate a multiple win around your project? 5

It is not just a lack of time that can prevent medical educators from engaging in scholarly evaluation efforts. Some educators may also feel inadequately trained in program evaluation and unclear what approaches and strategies to employ when engaging in program evaluation. Further, if evaluation is completed well, there is often an opportunity to translate this work into scholarly outputs.

The goal of this paper was to accomplish three goals: (1) to introduce educators to the concept of program evaluation, (2) to help them to understand frameworks that will guide them in correctly and rigorously performing program evaluations, and (3) to discuss ways in which program evaluation can translate to scholarly output.

WHAT IS PROGRAM EVALUATION?

In medical education, a “program” can refer to a large spectrum of activities, and experiences—they can range from a new workplace‐based assessment program 6 , 7 to a boot camp series 8 to a longitudinal faculty development course. 9 , 10 It is an ever‐evolving field with new technologies, shifting paradigms, and often unclear scholarly formats. The delivery of medical education requires the implementation of programs. Whether it is a well‐established program (e.g., intern orientation or airway management training) or a novel approach to assessment (e.g., simulation‐based critical care competency or entrustable professional activity), these programs need to be evaluated to determine if they are worthwhile with respect to effectiveness or value. A formal definition for program evaluation has been put forth by Mohanna and Cottrell as “a systematic approach to the collection, analysis, and interpretation of information about any aspect of the conceptualization, design, implementation, and utility of educational programmes”. 11 Simply stated, program evaluation is the process of identifying the value of an educational offering, but at times it can also be a way of determining issues or problems in need of systematic improvement.

Methods similar to those employed by experimentalists or epidemiologists may be used for measurement and analysis when conducting program evaluation, but this process is distinct from conventional research studies. Experimental research typically focuses on the generation of new knowledge that adds to the world more transferable or generalizable to other contexts, whereas program evaluation seeks to understand the efficacy of a specific, discrete project (e.g., a curricular change in a program or a new course design). Quantitative experiments may involve hypothesis testing with a control group and an experimental group, while qualitative studies may seek to understand or describe an experienced phenomenon. Despite being distinct from research, program evaluation is a rigorous process that might use a variety of quantitative and/or qualitative data to determine the value of the outcomes of a program, though technically a research protocol is not required.

WHY AND WHEN TO USE PROGRAM EVALUATION

While the specific purposes of program evaluation are extensive, at its core, program evaluation is about values, judgements, decision making, and change. 1 , 12 , 13 Program evaluation is another way, outside of the program itself, that you can create a value proposition to your community via your program. 14 Educators use program evaluation to determine the value and worth of the program they designed and then explain that worth to others. There are multiple program evaluation frameworks, and which framework you select is determined by the stakeholders and focus of the evaluation. 13 , 15

The ultimate why of your program evaluation will be how you define success of the program in the eyes of the stakeholders and the focus of the evaluation. 16 This marker of success should fall into at least one broad category of program evaluation—accountability, knowledge, or development—though these categories are often intertwined. 1 , 12 , 17 More specific purposes for evaluation within these three categories are found in Table  1 .

Purposes for program evaluation

Although it can resemble research (e.g., experimental or qualitative medical education research), it is differentiated from research by the fundamental underlying impetus for the study—research work seeks to understand the world better through its conduct (to create generalizable or transferrable “truths” to better understand how things work), whereas program evaluation seeks to understand how and if a specific program works.

If done correctly, program evaluation is a systematic method of answering questions about the program you have designed, providing insights for others to replicate or avoid in their own programs. 18 Once the work has been done, “dissemination to the community at large constitutes a critical element of scholarship.” 13 Dissemination of this work could be publishing the program evaluation as an original research report, as an innovation report, or in an online curricular repository (e.g., MedEdPORTAL, JETem) to help advance knowledge for others (Table  2 ).

Comparing and contrasting of various types of program evaluation scholarship

Overall, once the rationale is determined, program evaluation can be divided up into two groups that help direct the when —formative (i.e., used to improve the performance of the program, program monitoring, happens at various times) and summative (i.e., used for overall judgements about the program and its developers, usually at the end of the program). 19 , 20 No matter what the why , all programs should have program evaluations built into them. In fact, Woodward argues that program evaluation should be done within every part of the educational intervention process. For example, a needs assessment is the program evaluation determining the need for the program. 19 Ideally, the program evaluation should be developed alongside the program itself ensuring that one does a credible evaluation answering all required questions. 18 Early program evaluation development prevents later problems and allows data to be collected, as suggested by Durning et al., 16 during three phases: (1) before (establish a baseline and helps show how much of the outcomes are due to the program itself), (2) during (process measurements; allows developers to notice and fix problems early), and (3) after the program (outcome measurements). The why and when of program evaluation feed directly into the approach you take in doing the program evaluation (i.e., how you actually do this).

HOW TO USE PROGRAM EVALUATION METHODOLOGIES

As stated above, development of the program evaluation should happen alongside development of the program itself, meaning prior to launching the program (or the most recent class of participants). This involves identifying the specific goals of the evaluation by considering the potential stakeholders and end‐users of the resultant evaluation. With this information, educators can better align the breadth and focus of the evaluation with their specific needs (Box  1 ).

Components of a program evaluation

• Develop an evaluation question based on specific goals of various stakeholders
• Identify your theory of change
• Perform a literature search
• Identify your (validated) collection instrument
• Consider your outcomes with a broad lens

Once you have identified the target audience, next determine the underlying theory for change. The three most common theories for this are reductionism, system theory, and complexity theory. Reductionism relies upon an assumption that there is a specific order with a direct cause and effect for each action. 21 This approach, reflected in models such as the Logic model, 22 , 23 suggests that there is a clear linearity and predictable impact from each intervention. 1 System theory builds upon this with its roots in the general system theory applied to biology. 24 In this model, it is proposed that the whole of a system is greater than the sum of its individual parts. 24 Therefore, education programs expand beyond merely isolated parts, instead comprising the integration of the specific program components with each other and with the broader educational environment. Complexity theory expands further to adapt to the ever‐changing, more complex state of programs in real life. 1 , 25 There are multiple complex factors that can influence education programs, including the participants, influence of stakeholders and regulators, professional practice patterns, the surrounding environment, and expanding knowledge within the specific field, as well as with regard to the education concepts being taught. 1 Understanding the underlying theories can help inform the conceptual frameworks selected for evaluation, but we will dive into this more in the next section.

CONCEPTUAL FRAMEWORKS

There are many frameworks that can guide your program evaluation process. A full description of each of these is beyond the scope of this paper; however, our authorship team has detailed six program evaluation frameworks that have been featured in medical education (and specifically AEM Education and Training) including: CIPP, Kirkpatrick Model, Logic Model, Realist Evaluation, RE‐AIM, and SQUIRE‐EDU. Table  3 provides a description of some of the more commonly used frameworks and sources of further information on each of them. 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47

Conceptual frameworks for program evaluation

When creating the program evaluation, you may utilize frameworks to guide the data collection. The selection process for your conceptual framework will require consideration of the end‐users and which data will be most valuable to them. You should perform a thorough literature search to identify similarities and differences with prior programs. Questions should seek to assess the benefits and consequences of the new intervention or innovation. During the literature search, seek out existing tools used by similar programs to inform your evaluation tool design. Identify how this aligns with your current program evaluation needs and modify the tool where necessary. It is important to also collect validity evidence for your specific tool. 26 Even if a tool is “validated” in another setting, new validity should be sought for the current application within the context of the new program. 26 Since evaluation is often centered on a particular program, the evaluation plan may contain outcomes that are idiosyncratic rather than generalizable; however, best practices of questionnaire design should still be followed as much as possible (e.g., basing the tool on prior evaluation of a previous study, pilot testing a survey tool prior to launch to ensure readability and clarity).

Finally, consider the outcomes with a broader lens. While often considered with regard to learner‐oriented outcomes (e.g., Kirkpatrick model), it is also important to consider the costs (e.g., time, expenses, faculty) and broader societal implications as described further below. Those reading the findings will want to weigh the cost and benefits of the program.

MARKERS OF HIGH‐QUALITY PROGRAM EVALUATION

Program evaluation and research studies have very common features, depending on the objectives of a study, these two methods may become very similar. While research studies aim to produce new knowledge, program evaluation studies focus on the program quality and value. 27 When unsure, ethics boards guidelines are helpful for ensuring that the study that you are about to conduct is a program evaluation study. In the United States, many program evaluations will require institutional review board approval but are usually granted exemption status since program evaluations will fall well within normal educational practices. Ethics boards in Canada deem program evaluations exempt from the ethical review as per Tri___Council Policy Statement 2 [2018] Article 2.5. 28 Therefore, initially, a program evaluation study should be checked with the ethics board and receive an ethical exemption to make sure that the study purpose, objectives, data collection, and analysis aligns with it.

There are three common approaches to program evaluation studies: decision‐oriented, outcomes‐oriented, and expertise‐oriented. 29 In the previous section, various program evaluation frameworks and models were described that can yield to the overall approaches. These frameworks are of vital value to the overall program evaluation process. 1 Without using a framework, program evaluation may lose its focus and the flow of the study may become redundant and less helpful. As each framework focuses on different parts of a study, it is important for researchers to take into account the study’s objectives and focus. The face validity of a framework should be agreed by the investigators, meaning the outcomes of the study could be achieved through the selected framework. 13 A study could focus on many objectives such as trainees’ learning, satisfaction, and the intervention’s success in reaching various audiences. 1

Innovation reports are an integral part of program evaluation studies as they evaluate novel approaches to teaching and learning. Hall and colleagues reviewed the literature on the quality markers of innovation reports and came up with 34 items resulting in seven themes from analysis of the problem to dissemination of results to ensure that the innovation reports adequately provide insights and reproducibly. 30 Therefore, ensuring that a program evaluation study has rigor and reproducibility is very important for any type of program evaluation study. Box 2 provides various pearls to help researchers who will tackle to program evaluation studies. Box 3 contains an annotated bibliography that summarizes key resources for further reading.

Pearls for those interested in conducting program evaluation work

Based on prior literature on innovation reports and program evaluations, we have identified some common problems encountered when authors claim to have conducted these formats of studies:

Pearl 1: Plan the program evaluation from the onset . Ideally, program evaluation should be established prior to the program launch (or at least prior to the most recent cohort). Performing program evaluation once the program is ongoing will limit the available information and increase the risk of recall bias.

Pearl 2: Consider all of the inputs and outputs. The evaluators will need to think beyond just the learner outcomes and consider the broader outcomes, impacts, and the resources and requirements to run the program.

Pearl 3: Attempt to identify unintended outcomes. Intended outcomes are often tracked but a systematic inquiry into identifying unintended outcomes is often overlooked.

Pearl 4: Involve a statistician or a data scientist early. Some program evaluation approaches require complex statistical analysis and even further data exploration to understand complex data to be collected through the program implementation. A statistician or a data scientist can provide different approaches on how to analyze data and understand the relationship on program focus and outcomes.

Pearl 5: Chart the overall program evaluation process. Program evaluation could be very complex from planning to evaluation. Each step of the program evaluation should be represented with a figure in the study. This charting process will give readers a clear idea about the program evaluation steps and how the framework was implemented at each step.

Key resources for further reading

The following are key papers on the program evaluation methodology recommended for those interested in learning more.

1. Frye AW, Hemmer PA. Program evaluation models and related theories: AMEE guide no. 67. Med Teach . 2012;34(5):e288‐e299.

This is a review of several common program evaluation models and the benefits and limitations of each. The paper also provides examples of how to apply these in practice.

2. Cook DA. 2010. Twelve tips for evaluating educational programs. Med Teach . 32:296–301.

A concise article that breaks down program evaluation into twelve “tips” to guide the development and implementation. Not meant to be used alone, but again a solid introduction to the process with an included blank table for readers to start brainstorming their own program evaluations.

3. Goldie J. AMEE Education Guide no. 29: Evaluating Educational Programs. Med Teach . 2006; 28(3): 210–224.

An introductory how‐to guide for program evaluation of educational programs in general including the history and the process. A solid starting point for someone who is unfamiliar with the process and a solid introduction to allow better integration of the information provided in the AMEE no. 67 (included below) which walks the reader through theories to use as frameworks for their program evaluations.

4. Durning SJ, Hemmer P, Pangaro LN. The Structure of Program Evaluation: An Approach for Evaluating a Course, Clerkship, or Components of a Residency or Fellowship Training Program. Teach Learn Med , 19:3, 308–318, 10.1080/10401330701366796

While the other articles included here involve program evaluation in general, this article focuses on applying program evaluation to graduate medical education. While it is just one particular framework out of many that are available, it provides insight into how to apply program evaluation to programs that don’t necessarily fit the usual educational program mold. For medical educators beginning their program evaluation journey, having this example will allow them to see how other frameworks might be used for their programs.

Program evaluations can be seen as a gateway towards other forms of scholarship for those who are most at home developing programs and curricula. However, it should be acknowledged as its own form of scholarship that is unique and separate from curriculum development or research.

CONFLICTS OF INTEREST

Dr. Shera Hosseini has received funding for her postdoctoral fellowship from the McMaster Institute for Research in Aging (MIRA). Dr. Yilmaz is the recipient of a 2019 TUBITAK Postdoctoral Fellowship grant. Dr. Shah—none; and no grants. Dr. Gottlieb holds grants for unrelated work with the Centers for Disease Control and Prevention, Council of Residency Directors in Emergency Medicine, Society for Academic Emergency Medicine, and eCampus Ontario. Dr. Stehman—none, Dr. Hall—holds grants for unrelated work from the Royal College of Physicians and Surgeons of Canada, Queen’s University Center for Teaching and Learning, and the Physician Services Incorporated Foundation. Dr. Chan holds grants for unrelated work from McMaster University, the PSI foundation, Society for Academic Emergency Medicine, eCampus Ontario, the University of Saskatchewan, and Royal College of Physicians and Surgeons of Canada.

Hosseini S, Yilmaz Y, Shah K, et al. Program evaluation: An educator's portal into academic scholarship . AEM Educ Train . 2022; 6(Suppl. 1) :S43–S51. doi: 10.1002/aet2.10745 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]

Supervising Editor: Dr. Susan Promes

Assessment and Evaluation

What is it .

" Assessment  is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. When it is embedded effectively within larger institutional systems, assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher education." (Thomas Angelo, AAHE Bulletin, November 1995, p. 7)

Program-level assessment is NOT about evaluating individual students or faculty teaching effectiveness. Rather, it is about examining student performance and experience across a cohort of students and using the information to continuously improve curriculum effectiveness.

Why Engage?

Program-level assessment  is an opportunity...

• to discover whether students are learning in the ways we hope and expect; to understand, verify, or strengthen student learning and experience in the program;
• to identify curricular and pedagogical features and areas  that are working well and are in need of improvements;
• for creating rich conversations around student learning, pedagogy, and curriculum.
• to inform how resources should be allocated (for improvement, for continuity, for strengthening, etc.) and to plan for the future . 

Our distinguished commitment to teaching excellence, student learning, and continuous improvement drive the assessment process. 

Berkeley Assessment Stories

In a recent published book chapter (Envisioning Scholar-Practitioner Collaborations: Communities of Practice in Education and Sport, 2018), Tony Mirabelli (Assistant Director, Athletic Study Center)  and Kirsten Hextrum described the Athletic Study Center's iterative evolution of their program evaluation and improvement efforts. We interviewed Tony about the book chapter and what motivated him to initiate and integrate program evaluation in ASC.  [ Read more ] 

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Practice-based program evaluation in higher education for sustainability: a student participatory approach.

1. Introduction

2. state of the art: learning competencies and participatory approaches, 2.1. competencies for sustainability: an introduction, 2.2. applying competencies to the program level, 2.3. transdisciplinarity and participatory approaches to learning, 2.4. the participatory evaluation of learning programs, 3. methods: the participatory evaluation of a master program, 4. research results, 4.1. competencies delivered by the program: a student-led evaluation, 4.2. competencies delivered through the student-led, practice-based evaluation, 5. conclusions and discussion, author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest.

• UNESCO. Framework for the Implementation of Education for Sustainable Development (ESD) beyond 2019 ; UNESCO: Paris, France, 2019; pp. 1–19. [ Google Scholar ]
• Lozano, R. Diffusion of sustainable development in universities’ curricula: An empirical example from Cardiff University. J. Clean. Prod. 2010 , 18 , 637–644. [ Google Scholar ] [ CrossRef ]
• Storey, M.; Killian, S.; O’Regan, P. Responsible management education: Mapping the field in the context of the SDGs. Int. J. Manag. Educ. 2017 , 15 , 93–103. [ Google Scholar ] [ CrossRef ]
• Egelund Holgaard, J.; Hadgraft, R.; Kolmos, A.; Guerra, A. Strategies for education for sustainable development—Danish and Australian perspectives. J. Clean. Prod. 2016 , 112 , 3479–3491. [ Google Scholar ] [ CrossRef ]
• Sahakian, M.; Seyfang, G. A sustainable consumption teaching review: From building competencies to transformative learning. J. Clean. Prod. 2018 , 198 , 231–241. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Kiely, R. Small answers to the big question: Learning from language programme evaluation. Lang. Teach. Res. 2009 , 13 , 99–116. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Evans, T.L. Competencies and Pedagogies for Sustainability Education: A Roadmap for Sustainability Studies Program Development in Colleges and Universities. Sustainability 2019 , 11 , 5526. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• O’Byrne, D.; Dripps, W.; Nicholas, K.A. Teaching and learning sustainability: An assessment of the curriculum content and structure of sustainability degree programs in higher education. Sustain. Sci. 2015 , 10 , 43–59. [ Google Scholar ] [ CrossRef ]
• Giangrande, N.; White, R.M.; East, M.; Jackson, R.; Clarke, T.; Salo, M.; Penha-lopes, G. A Competency Framework to Assess and Activate Education for Sustainable Development: Addressing the UN Sustainable Development Goals 4.7 Challenge. Sustainability 2019 , 11 , 2832. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Bergsmann, E.; Schultes, M.; Winter, P.; Schober, B.; Spiel, C. Evaluation of competence-based teaching in higher education: From theory to practice. Eval. Program Plann. 2015 , 52 , 1–9. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Redman, A.; Wiek, A.; Barth, M. Current practice of assessing students’ sustainability competencies: A review of tools. Sustain. Sci. 2021 , 16 , 117–135. [ Google Scholar ] [ CrossRef ]
• Brundiers, K.; Barth, M.; Cebrián, G.; Cohen, M.; Diaz, L.; Doucette-Remington, S.; Dripps, W.; Habron, G.; Harré, N.; Jarchow, M.; et al. Key competencies in sustainability in higher education—Toward an agreed-upon reference framework. Sustain. Sci. 2020 , 16 , 13–29. [ Google Scholar ] [ CrossRef ]
• Perez Salgado, F.; Abbott, D.; Wilson, G. Dimensions of professional competences for interventions towards sustainability. Sustain. Sci. 2018 , 13 , 163–177. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Rieckmann, M. Future-oriented higher education: Which key competencies should be fostered through university teaching and learning? Futures 2012 , 44 , 127–135. [ Google Scholar ] [ CrossRef ]
• Wiek, A.; Withycombe, L.; Redman, C.L. Key competencies in sustainability: A reference framework for academic program development. Sustain. Sci. 2011 , 6 , 203–218. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Brady, A.M. The Regime of Self-Evaluation: Self-Conception for Teachers and Schools. Br. J. Educ. Stud. 2016 , 64 , 523–541. [ Google Scholar ] [ CrossRef ]
• Wiek, A.; Bernstein, M.J.; Foley, R.W.; Cohen, M.; Forrest, N.; Kuzdas, C.; Kay, B.; Keeler, L.W. Operationalising Competencies in Higher Education for Sustainable Development. In Routledge Handbook of Higher Education for Sustainable Development , 1st ed.; Routledge: Milton Park, UK, 2015; pp. 241–260. [ Google Scholar ]
• de Haan, G. The BLK ‘21’ programme in Germany: A ‘Gestaltungskompetenz’-based model for Education for Sustainable Development. Environ. Educ. Res. 2006 , 12 , 19–32. [ Google Scholar ] [ CrossRef ]
• Silvius, A.J.G.; Schipper, R.P.J. Sustainability in Project Management Competencies: Analyzing the Competence Gap of Project Managers. J. Hum. Resour. Sustain. Stud. 2014 , 2 , 40–58. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Garcia, M.R.; Junyent, M.; Fonolleda, M. How to assess professional competencies in Education for Sustainability? An approach from a perspective of complexity. Int. J. Sustain. High. Educ. 2017 , 18 , 772–797. [ Google Scholar ] [ CrossRef ]
• Shephard, K.; Harraway, J.; Lovelock, B.; Mirosa, M.; Skeaff, S.; Slooten, L.; Strack, M.; Furnari, M.; Jowett, T.; Deaker, L. Seeking learning outcomes appropriate for ‘education for sustainable development’ and for higher education. Assess. Eval. High. Educ. 2015 , 40 , 855–866. [ Google Scholar ] [ CrossRef ]
• Sandri, O.; Holdsworth, S.; Thomas, I. Vignette question design for the assessment of graduate sustainability learning outcomes. Environ. Educ. Res. 2018 , 24 , 406–426. [ Google Scholar ] [ CrossRef ]
• Cebrián, G.; Segalàs, J.; Hernández, À. Assessment of sustainability competencies: A literature review and future pathways for ESD research and practice. Cent. Eur. Rev. Econ. Manag. 2019 , 3 , 19–44. [ Google Scholar ] [ CrossRef ]
• Cebrián, G.; Junyent, M.; Mulà, I. Competencies in Education for Sustainable Development: Emerging Teaching and Research Developments. Sustainability 2020 , 12 , 579. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Barth, M.; Rieckmann, M. State of the Art in Research on Higher Education for Sustainable Development. In Routledge Handbook of Higher Education for Sustainable Development ; Routledge: Milton Park, UK, 2016; pp. 100–113. [ Google Scholar ]
• Borch, I.; Sandvoll, R.; Risør, T. Discrepancies in purposes of student course evaluations: What does it mean to be “satisfied”? Educ. Assess. Eval. Account. 2020 , 32 , 83–102. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Hubball, H.; Pearson, M.L.; Clarke, A. SoTL inquiry in broader curricular and institutional contexts: Theoretical underpinnings and emerging trends. Teach. Learn. Inq. 2013 , 1 , 41–57. [ Google Scholar ] [ CrossRef ]
• Lozano, R.; Barreiro-gen, M.; Lozano, F.J.; Sammalisto, K. Teaching Sustainability in European Higher Education Institutions: Assessing the Connections between Competences and Pedagogical Approaches. Sustainability 2019 , 11 , 1602. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Foucrier, T.; Wiek, A. Training Future Entrepreneurs—Developing and Assessing Sustainability Competencies in Entrepreneurship Education, 2020 ; Arizona State University: Tempe, AZ, USA, 2020. [ Google Scholar ]
• Holdsworth, S.; Sandri, O.; Thomas, I.; Wong, P.; Chester, A.; McLaughlin, P. The use of the theory of planned behaviour to assess graduate attributes for sustainability. Environ. Educ. Res. 2020 , 26 , 275–295. [ Google Scholar ] [ CrossRef ]
• Blackstock, K.L.; Kelly, G.J.; Horsey, B.L. Developing and applying a framework to evaluate participatory research for sustainability. Ecol. Econ. 2007 , 60 , 726–742. [ Google Scholar ] [ CrossRef ]
• Di Giulio, A.; Defila, R. Enabling university educators to equip students with inter- and transdisciplinary competencies. Int. J. Sustain. High. Educ. 2017 , 18 , 630–647. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Mobjörk, M. Consulting versus participatory transdisciplinarity: A refined classification of transdisciplinary research. Futures 2010 , 42 , 866–873. [ Google Scholar ] [ CrossRef ]
• Brundiers, K.; Wiek, A.; Redman, C.L. Real-world learning opportunities in sustainability: From classroom into the real world. Int. J. Sustain. High. Educ. 2010 , 11 , 308–324. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Balsiger, J. Transdisciplinarity in the class room? Simulating the co-production of sustainability knowledge. Futures 2015 , 65 , 185–194. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Caniglia, G.; John, B.; Kohler, M.; Bellina, L.; Wiek, A.; Rojas, C.; Laubichler, M.D.; Lang, D. An experience-based learning framework: Activities for the initial development of sustainability competencies. Int. J. Sustain. High. Educ. 2016 , 17 , 827–852. [ Google Scholar ] [ CrossRef ]
• Sipos, Y.; Battisti, B.; Grimm, K. Achieving transformative sustainability learning: Engaging head, hands and heart. Int. J. Sustain. High. Educ. 2008 , 9 , 68–86. [ Google Scholar ] [ CrossRef ]
• Lance Hogan, R. The historical development of programme evaluation: Exploring the past and present. Online J. Work. Educ. Dev. 2007 , II , 1–10. [ Google Scholar ]
• Strober, E. Is power-sharing possible? Using empowerment evaluation with parents and nurses in a pediatric hospital transplantation setting. Hum. Organ. 2005 , 64 , 201–210. [ Google Scholar ] [ CrossRef ]
• Holm, T.; Sammalisto, K.; Grindsted, T.S.; Vuorisalo, T. Process framework for identifying sustainability aspects in university curricula and integrating education for sustainable development. J. Clean. Prod. 2015 , 106 , 164–174. [ Google Scholar ] [ CrossRef ]
• Lozano, R.; Merrill, M.Y.; Sammalisto, K.; Ceulemans, K.; Lozano, F.J. Connecting competences and pedagogical approaches for sustainable development in higher education: A literature review and framework proposal. Sustainability 2017 , 9 , 1889. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• UNIGE Schedules and Regulation—Faculté des Sciences de la Société —UNIGE. Available online: https://www.unige.ch/sciences-societe/formations/masters-in-english/standardization/schedules-and-regulation/ (accessed on 1 August 2021).
• White, M.A. Sustainability: I know it when I see it. Ecol. Econ. 2013 , 86 , 213–217. [ Google Scholar ] [ CrossRef ]
• Aguado, N.A. Teaching Research Methods: Learning by Doing. J. Public Aff. Educ. 2009 , 15 , 251–260. [ Google Scholar ] [ CrossRef ]
• Barth, M.; Godemann, J.; Rieckmann, M.; Stoltenberg, U. Developing key competencies for sustainable development in higher education. Int. J. Sustain. High. Educ. 2007 , 8 , 416–430. [ Google Scholar ] [ CrossRef ] [ Green Version ]

Share and Cite

Curtis, H.L.; Gabriel, L.C.; Sahakian, M.; Cattacin, S. Practice-Based Program Evaluation in Higher Education for Sustainability: A Student Participatory Approach. Sustainability 2021 , 13 , 10816. https://doi.org/10.3390/su131910816

Curtis HL, Gabriel LC, Sahakian M, Cattacin S. Practice-Based Program Evaluation in Higher Education for Sustainability: A Student Participatory Approach. Sustainability . 2021; 13(19):10816. https://doi.org/10.3390/su131910816

Curtis, Helen Lee, Lucas Catalani Gabriel, Marlyne Sahakian, and Sandro Cattacin. 2021. "Practice-Based Program Evaluation in Higher Education for Sustainability: A Student Participatory Approach" Sustainability 13, no. 19: 10816. https://doi.org/10.3390/su131910816

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Update 22/08/2024: Due to technical disruption, we are experiencing some delays to publication. We are working to restore services and apologise for the inconvenience. For further updates please visit our website

Program Evaluation Pragmatic Methods for Social Work and Human Service Agencies

• Library eCollections

Description

Be prepared for your future role in a service-oriented agency. This textbook provides practical guidance on program evaluation while avoiding replicating other course material. Drawing on over 40 years of subject knowledge, Allen Rubin describes outcome designs that are feasible for service-oriented agencies and that match the degree of certainty needed by key users of outcome evaluations. The utility and easy calculation of within-group effect sizes are outlined, which enhance the value of evaluations that lack control groups. Instructions are…

• Add to bookmarks
• Download flyer
• Add bookmark
• Cambridge Spiral eReader

Key features

• Focuses on the practical aspects of performing evaluation tasks as service providers or administrators in human service agencies
• Balances the need for methodological rigor while maintaining realistic expectations consistent with becoming a successful evaluator within service-oriented settings
• Each chapter gives examples, summaries, and questions that instructors can use in class discussion
• Includes instructions on how to write and disseminate an evaluation report in a way that maximizes its chances of being used
• Gives advice on: surveys of client needs or satisfaction, monitoring program implementation of new agency initiatives, and maximizing the value of evaluations that have limited internal validity

About the book

• DOI https://doi.org/10.1017/9781108870016
• Subjects Health and Clinical Psychology, Psychology, Social policy and social work, Sociology
• Publication date: 23 July 2020
• ISBN: 9781108835992
• Dimensions (mm): 253 x 177 mm
• Weight: 0.58kg
• Contains: 11 b/w illus. 12 tables
• Page extent: 264 pages
• Availability: Available
• ISBN: 9781108799096
• Weight: 0.45kg
• Publication date: 18 September 2020
• ISBN: 9781108870016

Access options

Review the options below to login to check your access.

Personal login

Log in with your Cambridge Higher Education account to check access.

Purchase options

There are no purchase options available for this title.

Have an access code?

To redeem an access code, please log in with your personal login.

If you believe you should have access to this content, please contact your institutional librarian or consult our FAQ page for further information about accessing our content.

Allen Rubin has been teaching courses on program evaluation for over 40 years. He is the Kantambu Latting College Professor of Leadership and Change at the University of Houston's Graduate College of Social Work, past president of the Society for Social Work and Research, and a fellow in the American Academy of Social Work and Social Welfare.

Related content

AI generated results by Discovery for publishers [opens in a new window]

• Lara J. Farrell ,
• Thomas H. Ollendick ,
• Peter Muris

Online publication date: 12 April 2019

Big Data-Based Evaluation of Higher Education: Model Construction and Practice Path

• Research Article
• Published: 29 August 2024
• Volume 1 , pages 171–177, ( 2024 )

Cite this article

• Shunping Wei 1 ,
• Wenting Hou 1 &
• Fengjuan Jiang 1  

After the Overall Plan for Deepening the Reform of Education Evaluation in the New Era has been released for over two years, the reform of education evaluation has achieved a good start and important phased outcomes. Promoting the digital transformation of education evaluation and developing Big Data-based education evaluation are the main measures of current evaluation reform. Based on the case study of the Minzu University of China, this paper systematically sorts out the relevant research, constructs the factor model and process model of Big Data-based education evaluation from the perspectives of factors and process of evaluation, puts forward the application idea of Big Data-based education evaluation from the perspectives of full business, full process and full factors, and puts forward the practical path of Big Data-based education evaluation from the aspects of application traction, teacher training and safe operation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Explore related subjects

• Artificial Intelligence
• Digital Education and Educational Technology

Central Committee of the Communist Party of China, & China’s State Council. (2020). Overall plan for deepening the reform of education evaluation in the new era . (in Chinese).

Google Scholar  

Li, J., & Wu, S. (2022). Empowering smart campus to run schools and upgrading quality. Ethnic Education of China , (5), 4–6. (in Chinese).

Liu, B., Yuan, T., Ji, Y., Liu, B., & Li, L. (2021). Intelligent technology enabling education evaluation: Connotation, overall framework, and practice path. China Educational Technology , (8), 16–24. (in Chinese).

Qiu, J., Feng, L., & Shu, F. (2021). An analysis of the status quo of international higher education evaluation research under the Big Data environment in the past ten years: Journal-based text mining. Journal of Modern Information , 41 (9), 4–11. (in Chinese).

Song, N., Zheng, Z., & Zhou, Y. (2021). On the reform of basic education evaluation in the new era from the perspective of Big Data. China Educational Technology , (2), 1–7. (in Chinese).

Tian, W., Yang, L., Xin, T., & Zhang, S. (2022). Technology-enabled monitoring and evaluation of education: State of the art and prospects. Chinese Journal of Distance Education , 42 (1), 1–11. (in Chinese).

Zheng, Y., & Liu, H. (2015). Path analysis of the application of Big Data for education evaluation in the united states. China Educational Technology , (7), 25–31. (in Chinese).

Zhu, C., & Yan, G. (2018). Modernization and specialization: The logics of new technology advance in educational evaluation in the era of Big Data. Tsinghua Journal of Education , 39 (5), 75–80. (in Chinese).

Zhu, D., & Ma, X. (2019). New technology promotes specialization: The logic of educational evaluation reform in the era of Big Data. Tsinghua Journal of Education , 40 (1), 5–7. (in Chinese).

Download references

Acknowledgments

This study was funded by the National Natural Science Foundation of China (Grant No. 72274234) and the 2023 First-Class Undergraduate Course Construction Project of Minzu University of China.

Author information

Authors and affiliations.

School of Education, Minzu University of China, Beijing, 100081, China

Shunping Wei, Wenting Hou & Fengjuan Jiang

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Shunping Wei .

Ethics declarations

Conflict of Interest The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Wei, S., Hou, W. & Jiang, F. Big Data-Based Evaluation of Higher Education: Model Construction and Practice Path. Front. Digit. Educ. 1 , 171–177 (2024). https://doi.org/10.1007/s44366-024-0006-y

Download citation

Received : 10 November 2023

Accepted : 26 December 2023

Published : 29 August 2024

Issue Date : June 2024

DOI : https://doi.org/10.1007/s44366-024-0006-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• education evaluation
• evaluation model
• evaluation system
• practical path
• Find a journal
• Publish with us
• Track your research

• International Applicants

MS in Educational Administration: Higher Education Student Affairs

Join our community.

School code

Entry Level Courses

Up to 10 credits in graduate-level coursework in education or related fields may be transferred in prior to matriculating into the Master’s degree program if approved by the faculty advisor. These transfer credits can fulfill an elective requirement or be a direct substitution of a required course. Required courses that are part of accreditation assessments cannot be substituted.

Masters' Core

ED406 Master's Research Methods

Specialization Courses

ED476 Administration of Student Affairs in Higher Education

ED485 College Students and Student Development Theory

EDU576 Contemporary Issues in Higher Education

ED437 Diversity and Equity in Higher Education

ED436 How Universities Work

ED475 Leadership and Management in Higher Education

EDU485 College Access and (In)Equity

EDE479 Assessment, Accreditation and Accountability in Higher Education

EDU490 Higher Education Law

EDU492 Governance, Policy, and Administration of Higher Education

EDU493 History of Higher Education

EDU496 Fiscal Issues in Higher Education

ED433 Student Affairs Administration: Admissions and Financial Aid

EDE485 Student Affairs Administration: Student Activities and Fraternity/Sorority Affairs

ED467 Student Affairs Administration: International Student Affairs

ED484 Student Affairs Administration: Residential Life

ED413 Student Affairs Administration: Academic Support Services

ED434 Student Affairs Administration: Minority Student Affairs

EDE487 The Role and Function of the American Community College in Higher Education

EDU454 Career Counseling and Development

EDU416 Understanding and Managing Conflict in Professional Organizations

ED430 College Retention: Theory, Research, and Practice

ED432 Professional Writing and Communications

EDE435 Service-Learning, Higher Education, and the Public Good

ED483 Communication and Counseling Skills for Teachers, Administrators, and Other Helping Professionals

EDU515 Decision Making for Educational Leaders I: Analyzing Problems in Schools and Universities

ED520 Program Evaluation

ED479 Human Capital Management in Higher Education

ED482 Technology and Higher Education

EDU446 Entrepreneurial Skills for Educators

Internships

If student works full-time, this may be waived by advisor with approval from Educational Leadership Chair, and substituted with an additional elective course.

EDF497 Supervised Internship in Higher Education (variable credits)

Additional Requirements

Master's students are required to complete a culminating experience as part of their degree program. This requirement may be met by completing a Master's Paper (for three credits as EDE 461 or zero credits) or Culminating Exam (zero credits). If meeting this requirement for zero credits, you must take another elective.

EDE461 Master's Culminating Requirement: Higher Education

Contact admissions

(585) 275-3950

[email protected]

Request information

Ready to apply?

Take a course before applying.

Higher Education - Ph.D.

• Colleges & Schools
• Undergraduate Catalog
• Graduate Catalog
• Minors List
• Health Sciences Career Tracks
• Request Information
• Search Careers

Degree Information

Start your future with Tennessee Tech!

Degree details.

The online Ph.D. in Higher Education is designed for professionals in the field who are interested in using data science to best prepare students and confront challenges facing higher education. The program will provide preparation to be adapted to any post-secondary role with a focus to improve college access, student success, and persistence to completion. Driven by data science, program candidates will apply actionable approaches to face challenges confronting higher education. Graduates will use their data science knowledge and skills to guide colleges and universities, state higher education agencies, foundations, and related associations.

Admissions Requirements

Students seeking admission to the Higher Education Ph.D. program are required to have:

• Degree:  a Master’s degree from an accredited institution
• GPA:  at least 3.0 on a 4.0 scale
• Professional Curriculum Vitae (CV) / Resume
• Letters of Recommendation:  Two letters of recommendation that speak to the individual’s scholarly aptitude and capability, particularly with respect to graduate-level work
• Statement of Intent: that speak to the individual’s scholarly aptitude and capability, particularly with respect to graduate-level work
• Academic Writing Sample: a sample that demonstrates excellent scholarly writing ability (applicant must be sole author)
• Interview:  must interview at the discretion of the Higher Education Ph.D. application review committee.

International students are required to demonstrate competency in spoken and written English by taking appropriate standardized tests. See accepted standardized tests and minimum required scores »

Career Opportunities

Learn More About Career Development

Learn more about next steps and more through the College of Graduate Studies.

Explore majors and programs of study

College of Graduate Studies

How to Apply

Explore Tech and let us share why your bold future begins here!

Have questions? Graduate Studies will help answer any questions you may have.

Paying for College

Paying for college is easier than you might think!

Connect on Social Media

Study what you love with expert faculty and hands-on learning!

Quick Links

• Tech at a Glance
• Majors & Concentrations
• Student Life
• Research at Tech
• Tech Express
• Current Students
• Faculty & Staff
• Mission and Vision
• Facts about Tech
• University Rankings
• Accreditation & Memberships
• Maps & Directions
• Board of Trustees
• Office of the President
• Strategic Plan
• History of Tech
• Parents & Family
• International
• Military & Veteran Affairs
• Tuition & Fees
• Financial Aid
• Visit Campus
• Scholarships
• Dual Enrollment
• Office of the Provost
• Academic Calendar
• Volpe Library
• Student Success Centers
• Honors Program
• Study Abroad
• Living On Campus
• Health & Wellness
• Get Involved
• Student Organizations
• Safety & Security
• Services for Students
• Upcoming Events
• Diversity Resources
• Student Affairs
• Featured Researchers
• Research Centers
• ttusports.com
• Social Media
• Student Resources
• Faculty & Staff Resources
• Bookstore/Dining/Parking
• Pay Online - Eagle Pay
• IT Help Desk
• Strategic Planning
• Office of IARE
• Student Complaints

A member of

Your path to success: fast 3-day credential evaluations, flat rate guarantee.

IEE offers fast 3-day credential evaluations with transparent, flat-rate pricing. Our evaluations come with unparalleled expert assistance from start to finish, ensuring a seamless application process.

Services and Fees

Document report this report identifies the institution(s) attended, dates of attendance, credential(s) earned, and the united states educational equivalent., education course this report identifies the institution(s) attended, dates of attendance, credential(s) earned, and the united states educational equivalent., translations this report identifies the institution(s) attended, dates of attendance, credential(s) earned, and the united states educational equivalent., not sure where to start.

Let us guide you through the process by selecting which service you need.

Not sure which service you need? Let's get you started within no time!

Get Your Credential Evaluation in 3 Easy Steps!

Our streamlined digital application means you stay informed every step of the way and receive your evaluation in 3 days or less!

Evaluation with the ‘ Experienced ’

To celebrate the thousands of lives changed over our 40 years in business, IEE launched 3 day standard processing time after receipt and approval of your documents. This means you are 3 days closer to that school acceptance letter, job offer or visa approval.

Helpful tools

Gpa calculator.

GPA calculations can be complex. Use our GPA calculator to estimate your US GPA.

Our GPA Calculator utilizes advanced algorithms to provide an accurate conversion of your international education to the U.S. GPA grading system, from almost any country in the world. Simply input your country and grading scale to see your results. Try it today!

Documentation

Get a list of required documentation for your evaluation instantly!

Use our documentation tool to simplify your evaluation process and the get-up-to-date requirements you will need. Simply input your home country to get your results!

Application Process

Getting your evaluation is easy with IEE. Our streamlined digital process means you stay informed every step of the way and receive your evaluation in 3 day or less!

Create an account

Upload documents, receive evaluation, track progress, unlock your potential: accurate credential evaluations for international students & professionals.

For over 40 years, we have been trusted by leading universities and employers globally. Join the ranks of thousands whose lives have been transformed by our credential evaluation services.

Learn from the experiences of real people who enjoyed the exceptional services with IEE.

My documents were evaluated very quickly and within 3 days, the support was excellent and I really appreciate the accuracy and speed of the process. Also, IEE gave me the opportunity to evaluate the provisional qualifications, for which I am very grateful. You are the best, thank you very much...

IEE EVALUATION IS simply Super. I like mainly customer support 24/7 always they respond any time and evaluation time also very less and low price. Tnq Iee..

Get an evaluation today

Join the thousands of people just like you who have made IEE their trusted evaluation service! We are here to answer your questions on chat, email and phone 24/7!

Course Report

also known as Course by Course Evaluation

This report identifies the institution(s) attended, dates of attendance, credential(s) earned, and the United States educational equivalent. It also lists individual courses taken with semester units, individual grade equivalents and an overall grade point average (where applicable). If you are planning on applying to multiple schools, this option is recommended

General Report

also known as General Diploma Evaluation

Identifies, describes, and provides a U.S. equivalency for each credential

Expedited Services

Standard delivery.

3 Business Days

Special Service

2 Business Days

Same day Service

if documents are received by 11 AM EST

Extra Copies

Additional hard copy.

When placing your order

After order completion

Emailed Copy

Iee portal delivery, delivery options, in office pick-up, shipping options.

Electronic Delivery to School

Domestic Priority

Domestic Express

Domestic Overnight

International Priority

Best-in-class Credential Evaluations

For Students & Immigrants

For Institutions & Employers

Founding Member

Over the past 50 years, we have led and shaped the credential evaluation field. As a founding member of the National Association of Credential Evaluation Services (NACES) and a member of the Alliance of Credential Evaluation Services of Canada (ACESC), we uphold ethical and fair standards in the assessment of international qualifications.

Why Choose WES?

Comprehensive Expertise

We have a diverse staff and a custom-built database
that contains over 46 years of research.

We have knowledge of institutions in

200+ Countries and territories

4M+ Trusted Evaluations

provided to international students and professionals

We evaluate knowledge of institutions from

48,000+ Institutions

Credential Evaluations for Students & Immigrants

Showcase your educational accomplishments with a verified report that is both accepted and respected by licensing boards, academic institutions, and employers throughout the U.S. and Canada. Your future starts here.

Select Destination Country

Do you know what kind of credential evaluation you need? Learn about your options and preview the potential cost of your evaluation report?

Learn about your options and preview the potential cost of your evaluation report.

What documents do you need to send WES? Enter information about your educational background to preview your list of required documents.

This free tool lets you see what your credentials might be equivalent to in Canada. Discover the value of applying for an official report from WES.

A WES Digital Badge is a web-enabled version of your credential evaluation, which means that it can be verified online–instantly. This gives you maximum visibility and recognition.

Our live and on-demand events are designed to assist international students and skilled immigrants in their quest to pursue an education or career in the Canada.

This free tool lets you see what your credentials might be equivalent to in the U.S. Discover the value of applying for an official report from WES.

See how your grades compare on the 4.0 grading scale used in the United States. Conversions are based on the most common grading scales for your country of education.

WES Career Pathways e-guides provide guidance and resources to internationally trained immigrants and refugees seeking to use their education and credentials in the U.S.

Our live and on-demand events are designed to assist international students and skilled immigrants in their quest to pursue an education or career in the U.S.

Credential Evaluations for Institutions & Employers

WES is the leading credential evaluation services provider in North America. We have provided 3 million applicants with best-in-class credential evaluations that are accepted by 2,500 employers, academic institutions, and regulators in the United States and Canada.

WES now partners with over 7,000 international institutions to securely transfer digital documents and expedite the evaluation process for our applicants.

Our secure online delivery system, AccessWES , is used to send evaluation reports to academic institutions and other organizations.

This free tool lets you preview the equivalency of international credentials in Canada.

Discover free international education resources for higher educational professionals.

Preview the list of documents that will be required to start an applicant’s evaluation report.

Our live and on-demand events are designed to assist institutional partners.

Quickly determine if an international student meets your eligibility requirements to enroll using WES iGPA Calculator for Institutions.

This free tool lets you preview the equivalency of international credentials in the U.S.

Our live and on-demand events are designed to assist our institutional professional partners.

For Refugees and Displaced Persons

WES strives to open pathways for individuals with international credentials. Individuals who have been displaced as a result of adverse circumstances in their country and have limited proof of academic achievements may be eligible for an alternative credential assessment through the WES Gateway Program.

“I used this service to evaluate my diplomas obtained in Nigeria. The site was easy to navigate and I found the information needed . Good customer staff and prompt response. Thank you!”

Opeyemi Oluwafemi WES Customer

“After many years where I could not make use of my education because I had foreign degrees , I can do it now because of an organization called World Education Services.”

Purnima Mensinkai WES Customer

Should you get a credential evaluation? That’s a big yes.

Ellen, Public Health Officer in Malawi

We have updated our Privacy Policy and Cookie Policy . By using this website, you accept the new terms.

• Bilingual Authorizations (CL-628b)
• Credentialing Information
• Applications, Forms and Leaflets

Bilingual Authorizations allow the holders to provide instruction to English Learners (EL). Assembly Bill (AB) 1871, signed by the Governor on September 30, 2008, provides for the issuance of bilingual authorizations rather than certificates, and expanded the options available to meet the requirements for the Bilingual Authorization. For a summary of all documents that authorize instruction to EL students, see the leaflet Serving English Learners, CL-622 . The section below lists the types of instruction authorized by Bilingual Authorizations. Each type of instruction is defined below.

Types of Instruction to English Learners Authorized by the Bilingual Authorization

• Instruction for English Language Development (ELD)
• Instruction for Primary Language Development
• Specially Designed Academic Instruction Delivered in English (SDAIE)
• Content Instruction Delivered in the Primary Language

Definitions of Types of Instruction

Instruction for English language development (ELD) means instruction designed specifically for EL students to develop their listening, speaking, reading, and writing skills in English. This type of instruction is also known as English as a Second Language (ESL) or Teaching English to Speakers of Other Languages (TESOL).

• Instruction for primary language development means instruction for EL students to develop their listening, speaking, reading, and writing skills in their primary language.
• Specially Designed Academic Instruction Delivered in English (SDAIE) means instruction in a subject area delivered in English that is specially designed to provide EL students with access to the curriculum.
• Content Instruction Delivered in the Primary Language means instruction for EL students in a subject area delivered in the students’ primary language.

Authorization

English Language Development (ELD) and instruction for primary language development are generally authorized at the level indicated by the holder’s prerequisite credential (see list of prerequisites in #1, below) including preschool, grades K-12 and in classes organized primarily for adults. With a Children’s Center Instructional Permit, Child Development Permit or a Children’s Center Supervision Permit, ELD and instruction for primary language development are limited to the programs authorized by the permits. With a Designated Subjects Teaching Credential in adult education, ELD and instruction for primary language development are limited to classes organized primarily for adults. With all prerequisite credentials or permits, Specially Designed Academic Instruction Delivered in English (SDAIE) and content instruction delivered in the primary language are authorized in the subjects and grade levels of the prerequisite credential or permit.

Requirements for the Bilingual Authorization

Individuals must satisfy all of the following requirements:

• Emergency Permits
• Provisional Internship Permits
• Short-Term Staff Permits
• District Intern Credentials
• University Intern Credentials
• Exchange Credentials
• Sojourn Certificated Employee Credentials
• Services credentials without a special class authorization
• Valid Language Development Specialist (LDS) Certificate, CLAD Certificate, teaching credential with an English Learner Authorization or CLAD Emphasis, or
• Passing scores on Subtests 1, 2 and 3 of the California Teacher of English Learners (CTEL) Examination. Scores used for certification purposes may be no older than ten years from the individual passed exam date.
• Possession of an out-of-state credential showing an English learner authorization.
• Possession of the National Board for Professional Teaching Standards certificate in Early and Middle Childhood/English as a New Language or Early Adolescence through Young Adulthood/English as a New Language.
• Passing scores on Tests II (or III depending on the language, covering Language and Communication), IV, and V of the California Subject Examinations for Teachers (CSET): World Languages. Scores used for certification purposes may be no older than ten years from the individual passed exam date.
• Completion of course work in a Commission-approved bilingual program. Once the program has been completed, the program sponsor must recommend the applicant for the Bilingual Authorization.
• Completion of course work in a Commission-approved bilingual program combined with passing scores on the CSET: World Languages Examination(s), based on equivalency as determined by a Commission-approved Bilingual Authorization program. For more information, see Coded Correspondence 10-07 .

Teachers who hold a valid, non-emergency California Single Subject or Standard Secondary Teaching Credential with a major in a language other than English need not take Test II or III of the CSET: World Languages Exam to qualify for a bilingual authorization in that language. Teachers who hold a three-year or higher degree from a foreign institution in which all instruction is delivered in a language other than English also do not need to take Test II or III to qualify for a bilingual authorization in that language. The foreign institution must be equivalent in status to a regionally-accredited institution of higher education in the United States.

Back to top

Examination Information

The CTEL examinations consist of the following three tests:

• CTEL 1: Language and Language Structure
• CTEL 2: Assessment and Instruction
• CTEL 3: Culture and Inclusion

The CSET: World Languages examinations used for bilingual authorizations consist of the following three tests:

• Test II (or III depending on specific language): Language and Communication (Listening, Speaking, Reading, Writing).
• Test IV: Bilingual Education and Bilingualism.
• Test V: Geographic, Historical, Sociopolitical, and Sociocultural Contexts.

Information regarding test dates, fees, and registration may be obtained by contacting the CSET program directly. See Table 1 for a list of the bilingual subtests by language.

CSET Program Evaluation Systems Group of Pearson 1224 N Market Boulevard Sacramento, CA 95834 http://www.ctcexams.nesinc.com Phone: (800) 205-3334 (toll free, US and Canada only) 9:00 a.m.–5:00 p.m. Pacific Time - M–F (excl. holidays) Automated information system (available 24 hours)

Assessor Agency for Portuguese

To earn a Bilingual authorization in Portuguese, individuals may take the CSET: World Languages Subtest IV (Bilingual Education and Bilingualism) and the Bilingual assessment administered by the following Commission-approved assessor agency:

California State University-Stanislaus 1 University Circle Turlock, CA 95382 (209) 667-3638 Contact: Dr. Elmano Costa CSU Stanislaus Center for Portuguese Studies [email protected]

Obtaining a Bilingual Authorization Based on a Previously Issued Comparable Document

Individuals holding a valid Bilingual Certificate of Competence (BCC) or BCLAD Certificate may apply to have a Bilingual Authorization added to any valid prerequisite credential or permit with the submission of an application ( form 41-4 ) and current application fee . Please note that a separate application and fee will be required for each document for which the authorization is requested. This action is not required. The BCC and BCLAD Certificate authorize the same service as the Bilingual Authorization and remain valid as long as the teacher’s prerequisite teaching credential remains valid.

Out-of-State Prepared Individuals with Bilingual Authorization

If an individual is applying for a Multiple Subject, Single Subject, or Education Specialist Teaching Credential and evidence of a full bilingual authorization from a state other than California is submitted with that application packet, there is no need to apply separately for the bilingual authorization; the bilingual authorization will be listed on the corresponding teaching credential. There is no need to submit a separate application and fee if applying via this method.

All other individuals who hold a bilingual authorization from a state other than California may apply directly to the Commission for a bilingual authorization by satisfying all of the following:

• Possess a valid California prerequisite credential.
• Satisfy the second-language requirement.
• Submit a copy of the out-of-state credential verifying a full bilingual authorization [1].
• Submit a completed application ( form 41-4 ).
• Submit full application processing fees .

Note [1]: See Table 2 at the end of this leaflet for more information on comparable out-of-state bilingual authorizations.

Applying for a Bilingual Authorization

Individuals that complete a Bilingual Authorization program or a combination of program course work and bilingual examinations must be recommended by their Commission-approved Bilingual Authorization program sponsor and complete the application and payment online. Individuals that pass all the required CSET: World Languages subtests and meet the requirements below may apply directly to the Commission for the Bilingual Authorization by submitting all of the following:

• Verify holding or eligibility for a CLAD Certificate or teaching credential with an English learner authorization (For more information, see leaflet CL-628C ).
• Original or photocopy of a CTEL Examination score report.
• Photocopy of an out-of-state credential showing a full English learner authorization.
• Original or photocopy of the CSET: World Languages Examination score report.
• Completed application ( form 41-4 ).
• Application processing fee .

Applicants requesting the bilingual authorization who hold more than one eligible document must designate the specific prerequisite document to which the authorization should be added. If more than one valid prerequisite document is held, it is not necessary to file an additional application or pay an additional fee, as the authorization on one document already applies to all other valid prerequisites held by the teacher. However, if an individual wants to add the authorization to more than one document, a separate application and fee will be required for each additional valid prerequisite document.

Table 1 CSET Subtests Required for the Bilingual Authorization

• The first subtest listed (either Subtest II or Subtest III, depending on the language) assesses language and communication skills. Note : the language and communication skills subtest that can be used toward a Bilingual Authorization is the same subtest for that content area as is used toward obtaining the Single Subject Teaching Credential in the corresponding language other than English.
• Subtest IV does not focus on any specific language (the same test applies to all languages) and assesses the methodology of bilingual education.
• Subtest V assesses bilingual cultural knowledge (the same test is used for Cantonese and Mandarin).

Click the table header to sort by that column. Type the Language name, or Requirement Type in the second row to filter that column.

Note [1]: For the Bilingual Authorization, passage of this subtest is not required if (1) you hold a valid, nonemergency Single Subject or Standard Secondary Teaching Credential with a major in the language authorized by the Bilingual Authorization you are seeking or (2) you hold a three-year or higher degree from a foreign institution in which all instruction is delivered in the language authorized by the Bilingual Authorization you are seeking and the institution is equivalent in status to a regionally accredited institution of higher education in the United States.

Guide to Out-of-State Bilingual Authorizations May 2017

Individuals who hold teaching credentials from another state with one of the authorizations listed in the following chart may request that the corresponding Bilingual authorization be listed on their California teaching credential. For additional information, see Coded Correspondence 17-05 regarding Bilingual authorizations. This is a guide and may not contain all acceptable Bilingual authorizations issued by each state. The Commission continues to research this information and changes will be made as additional information becomes available.

An individual who holds a document with a Bilingual authorization not included in the guide must provide a letter from the state agency that issued the document that verifies that the document issued authorized the holder to provide the equivalent English language development (ELD), specially designed academic instruction in English (SDAIE), primary language development, and content instruction delivered in the primary language services at the time the individual earned the document. Primary language development is defined as instruction for English learner students to develop their listening, speaking, reading, and writing skills in their primary language. Content instruction delivered in the primary language is defined as instruction for English learner students in a subject area delivered in the students’ primary language.

The Commission reserves the right to make the final determination regarding the equivalency of any authorization submitted for evaluation. 

Table 2 Out-of-State Bilingual Authorizations/Endorsements

Click the table header to sort by that column. Type the Sate name, or Authorization/ Endorsement Type in the second row to filter that column.

Educator Login

Application Status

We are currently processing applications received before: 6/19/2024

Credentials Links

• Create Your User ID/Log In Issues
• Change Your Name
• CTC Online Services Help
• Examinations
• Credentials FAQ - General Questions
• Fingerprint Information
• Transcript Guidance

Information

• About the Commission
• News and Events
• Commission at a Glance

• Search News and Media
• All of Oklahoma State University

Headlines News and Media

Osu completes ethiopian higher education program.

Tuesday, August 27, 2024

Media Contact: Hailey Williams | Communications Specialist | 405-744-5496 | [email protected]

Dr. Randy Kluver, associate provost and dean of Oklahoma State University Global , recently represented OSU at the closing workshop for the Ethiopian Higher Education Leadership, Management, and Governance program in Addis Ababa, Ethiopia.  

The LMG program is funded by the U.S. Embassy in Addis Ababa, the nation’s capital. It is a three-year project to strengthen higher education leadership in Ethiopia’s public universities by providing senior leaders with insight into global trends in higher education and to develop strategies to improve the quality of Ethiopian higher education.  

Along with Texas Tech University and Ohio State University, Oklahoma State partnered with the U.S. Department of State, the U.S. Embassy in Addis Ababa, the Ethiopian Ministry of Education and Ohio State Global One Health.  

The program is the result of a U.S. Department of State grant .  

Throughout the program, participants received training in Ethiopia. They participated in visits to the three partner universities to meet with leaders and learn how American institutions develop strategies to meet challenges in higher education leadership and administration.  

The closing workshop brought representatives from these organizations together with the program participants to reflect on its lasting impact on Ethiopia’s higher education.  

“With the university’s connections and history with Ethiopia, OSU Global was glad to play a part in this program,” Kluver said. “This workshop was a fantastic opportunity to connect with each other and review everything that has been learned over the course of the project.  

“I look forward to seeing the effects the LMG program will have on the future of Ethiopian higher education.”  

• About About collapsed link
• Community College Community College collapsed link
• Short-Term Training Program Short-Term Training Program collapsed link
• Champion Toolkit
• Press Releases
• Reconnect Data Dashboard
• Meet the Navigators
• Already Accepted? Keep going.
• Check Your Application Status
• Calculating Your Costs
• FAFSA Resources
• Frequently Asked Questions
• List of Colleges
• Stay Reconnected
• Michigan Reconnect for ages 21 to 24
• Michigan Reconnect para personas de 21 a 24 años.
• Veterans, Service Members and Their Families
• Denise Brown
• David Cuevas
• Jamar Ragland
• Allen Chung
• Taylor Dean
• VLecea V Hunter
• Nicole Reed
• Amber Rosenbrock
• Terrance Shorter Jr.
• Tracy Spilker
• Marsha Tompkins

College Checklist

Find your In-District Community College

Veterans, Services Members and Their Families

• Short-Term Training Program FAQs
• Community College
• Short-Term Training Program

Search is currently unavailable. Please try again later.

Popular on michigan.gov

• Agriculture and Rural Development
• Civil Rights
• Environment
• Health and Human Services
• Natural Resources
• Secretary of State

How Do I...

• Register to Vote
• Renew My License Plate
• View assistance programs

The web Browser you are currently using is unsupported, and some features of this site may not work as intended. Please update to a modern browser such as Chrome, Firefox or Edge to experience all features Michigan.gov has to offer. 

• Google Chrome
• Microsoft Edge

MiLEAP Partners with Delta Air Lines to Host Michigan Reconnect Enrollment Day for Employees, Provide Assistance in Applying for the Tuition-Free Scholarship Program

August 28, 2024

Collaboration highlighted opportunities for students to earn a postsecondary degree or credential while pursuing a path to advanced career mobility with Delta

The Michigan Department of Lifelong Education, Advancement and Potential (MiLEAP) joined Delta Air Lines at the Westin Detroit Metropolitan Airport recently to host a Michigan Reconnect Enrollment Day for employees. The event was designed to encourage Delta and partner employees to apply for Michigan Reconnect , a scholarship program that provides students the opportunity to earn a skills certificate or degree tuition-free at their local community college.

“Earning a postsecondary degree or skill certificate gives Michiganders a better chance at career advancement, security and a true path to prosperity,” said Dr. Beverly Walker-Griffea, director of MiLEAP . “By opening the door to Michigan Reconnect and encouraging opportunities to pursue higher education, Delta Air Lines is investing in its employees’ futures – ensuring they have the skills needed to fuel long-lasting careers in the airline industry and beyond.”

Delta has more than 100,000 employees, with approximately 4,700 working in Michigan. The company was recently ranked fifth in the nation as one of “America’s Best Large Employers” by Forbes, based on a survey of current and recent company employees.

During the event, representatives from Delta highlighted the importance of eliminating barriers for employees to access high-earning opportunities and fostering a diverse pipeline of talent within the company.

“Introducing our employees to Michigan Reconnect is yet another way Delta Air Lines is working to remove obstacles for advancement and strengthen the workforce pipeline of aviation professionals,” said Hussein Berry, Delta’s vice president of airport operations at Detroit Metropolitan Wayne County Airport. “Delta Air Lines prides itself on its ‘skills-first talent strategy’, which aims to enrich the careers of all Delta employees and accelerate career mobility by prioritizing a candidate’s skills and experience in internal and external hiring. Our skills-first hiring approach increases access to higher-earning positions while promoting equitable access to long-term career trajectories. Much like MiLEAP is removing barriers to education, Delta is removing obstacles to entry for talent.”

The enrollment day gave employees a chance to connect with Reconnect Navigators who offered insight on how to apply for the scholarship, which traditionally provides in-district community college tuition-free to Michiganders 25 and older. MiLEAP recently announced that adults ages 21 to 24 who have not yet earned a college degree now have until Dec. 31, 2024, to apply for Reconnect.

In addition to scholarship support, local community colleges were on-site to help students identify available programs that offer a path to growth in the airline industry and beyond, including areas such as business, aviation management, emergency services, skilled trades and more. Colleges and other partners also offered information on how Reconnect can help provide wraparound supports for issues like childcare, transportation and financial aid.

Partnerships like these are helping the state make progress toward its Sixty by 30 goal to increase the number of Michiganders with a postsecondary credential to 60% by 2030. Currently, 51.1% of adults in Michigan have a skill certificate or college degree.

For more information on Michigan Reconnect and eligibility, visit Michigan.gov/Reconnect.

About MiLEAP:

Established by Gov. Whitmer in 2023, MiLEAP’s mission is to improve outcomes from preschool to postsecondary so anyone can “make it in Michigan” with a solid education and a path to a good-paying job. To learn more about MiLEAP, go to Michigan.gov/MiLEAP .

Media Contact:

Aundreana Jones-Poole

Communications Manager

[email protected]

517-930-4919

Related News

Governor whitmer celebrates three-year anniversary of michigan reconnect, whitmer announces michigan reconnect milestone: over 100,000 applicants accepted into program, gilchrist highlights investment to help students overcome barriers to higher education, gov. whitmer highlights impact of tuition-free higher education programs in jackson, surrounding communities, gov whitmer highlights impact of tuition-free programs in macomb oakland and wayne counties, gov whitmer highlights impact of tuition-free programs in gr.

Official Site of The State of New Jersey

The State of NJ site may contain optional links, information, services and/or content from other websites operated by third parties that are provided as a convenience, such as Google™ Translate. Google™ Translate is an online service for which the user pays nothing to obtain a purported language translation. The user is on notice that neither the State of NJ site nor its operators review any of the services, information and/or content from anything that may be linked to the State of NJ site for any reason. - Read Full Dislaimer

• Search close

Governor Phil Murphy

ICYMI: New Student Loan Redemption Program to Support Home and Community-Based Workers

Human Services, Children and Families, and Higher Education Student Assistance Authority Unveil New Program to Attract & Retain Health Care and Social Services Professionals 

TRENTON – As part of a continued effort to improve and expand the home and community-based services workforce, the Department of Human Services, Department of Children and Families, and the Higher Education Student Assistance Authority today announced a new student loan redemption program to benefit health care, behavioral health, and social services professionals serving those with medical needs, behavioral and/or mental health conditions, and disabilities. Under the new Home and Community-Based Services Provider Loan Redemption Program, eligible workers can receive up to $50,000 in loan relief in exchange for one year of service at an approved home- and community-based services provider agency or as a self-directed employee. “Uplifting our healthcare workers is mission-critical to delivering on the needs of New Jersey residents. This student loan redemption program further bolsters our home and community-based services workforce, and it is key in supporting qualified service providers to bring their skills and expertise to communities across the state,” Governor Phil Murphy said. “In addition to alleviating the financial burdens of this workforce, this program also builds the capacity to deliver care in the community for more New Jerseyans. I applaud DHS Commissioner Sarah Adelman, DCF Commissioner Christine Norbut Beyer, and HESAA Executive Director Margo Chaly for their work to make this program a reality.” “We continue to invest in innovative approaches to strengthen and prioritize independence and person-centered care that will help individuals live in their own homes and remain active in their communities,” Human Services (DHS) Commissioner Sarah Adelman said. “This new student loan redemption program will benefit caregivers who provide vital supports to people with disabilities and with behavioral health needs, as well as older adults living in the community. Supporting our care workers must be a priority.” “At the Department of Children and Families, we are committed to a robust network of in-community and in-home services, so that families can stay together while they access the treatment or support solutions that best meet their needs,” Children and Families (DCF)  Commissioner Christine Norbut Beyer, MSW, said . “In partnership with DHS and HESAA, offering this loan redemption program is one part of a multi-level strategy to ensure that our contracted service providers can compete for the highly competent, well-trained workforce that New Jersey children, youth, and families deserve. These care workers help our families to heal from adversity and trauma, to strengthen the bonds of family and community, and to achieve their utmost potential – to be safe, healthy, and connected in all that they do.” “Our team is pleased to work in partnership with DHS and DCF to help bolster social services and health care workforces in New Jersey,” Higher Education Student Assistance Authority (HESAA) Executive Director Margo Chaly, Esq. said. “As with all student loan redemption programs that HESAA administers, the goal of this new program is to encourage professionals to start and continue working in high-need fields. By incentivizing eligible experts to serve in home and community-based settings, our state will ensure more residents can receive the mental health and social supports that they need.” Professionals eligible to apply include psychiatrists, licensed psychologists, licensed social workers, licensed clinical social workers, psychiatric nurse mental health clinical specialists, board certified behavior analysts, board certified behavior analysts-doctoral, licensed clinical or certified alcohol and drug counselors, licensed professional and associate counselors, licensed or associate marriage and family therapists, DCF care managers, registered nurses, and licensed practical nurses. Self-directed employees are also eligible to apply. Program funding comes from the Centers for Medicare and Medicaid Services and is part of a $100 million, FY24 strategic investment by DHS in home- and community-based services, including significant investments in workforce development. In addition to the student loan redemption program, the plan will establish recruitment, training, and certification programs for direct care staff, along with establishing new community-based housing options for individuals with disabilities or behavioral health conditions. The one-time allocation for this student loan redemption program provides:

• $5 million for eligible employees serving in DHS’ Division of Mental Health and Addiction Services -contracted agencies;
• $5 million for eligible employees serving in DHS’ Division of Developmental Disabilities -funded agencies and self-directing employees;
• $5 million for eligible employees at DCF-approved settings; and
• $2 million for eligible private duty nurses employed by agencies contracted by NJ FamilyCare ’s managed care organizations .

“We are proud to offer this new benefit to dedicated workers who support individuals with disabilities and older adults in the community,” DHS Deputy Commissioner for Aging and Disability Services Kaylee McGuire said. “Creative steps such as a loan redemption program will help attract and retain workers and build a stronger foundation for the future.” To qualify for this student loan redemption program, an applicant must meet all of these requirements:

• Be employed full-time in an eligible profession by a State-funded home- and community-based provider agency;
• Maintain a current license/board certification, as applicable, to practice in New Jersey throughout participation in the program;
• Maintain employment at the State-funded home- and community-based provider agency for at least one year, pursuant to the terms of a contract with HESAA;
• Not be in default on any eligible qualifying student loan; and
• Not currently be participating in any other student loan redemption program or in the federally-administered National Health Service Corps Loan Repayment Program.

This initiative builds upon efforts by the Murphy Administration to bolster this critical workforce, including funding wage increases for home health aides, certified nurse assistants, direct support professionals, child care workers, behavioral health providers, and more as the state’s minimum wage increased to $15 per hour over recent years, as well as launching a Jobs that Care New Jersey website and paid advertising campaign to promote direct care training and job opportunities. Provider agencies are encouraged to share information about this program with all eligible employees prior to July 1, when applications will open for a 30-day period. Eligible program participants who submit a complete and accurate application will be selected by October 1 on a first-come, first-serve basis. For more information on the program and to apply once the application opens, visit here .