Framework

Assessment Frameworks

INTRODUCTION

To comply with SACSCOC expectations, UTRGV’s educational programs, administrative support units, and student support units are expected to engage in the Institutional Effectiveness (IE) process. The process requires programs/units to cycle through the following steps:

Assess/Measure - Intervene - Re-assess/measure

The Office of Academic and Institutional Excellence (OAIE) has developed two assessment frameworks that allow programs/units to document the various elements of the IE process over time and demonstrate compliance with SACSCOC C.R. 7.3, C.R. 8.2.a and C.R. 8.2.c. These frameworks include, 1) a temporary two-year assessment framework titled the Transitional Assessment Framework (2015-2017) and 2) a continual five-year assessment framework titled the New Assessment Framework (2017-2022).

The Transitional Assessment Framework is intended to help UTRGV’s programs/units document their assessment activities immediately following the consolidation of its legacy institutions (fall 2015) until UTRGV’s Compliance Certification Report for SACSCOC Reaffirmation is submitted (fall 2018). Although UTRGV only recently celebrated its inaugural year, SACSCOC expects all programs/units to continuously engage in the Institutional Effectiveness process.

Beginning in the fall of 2017, the New Assessment Framework will go into effect. This continual five-year assessment framework extends the reporting expectations over a longer period of time to provide more ownership and flexibility for programs/units engaging in assessment. Additionally, the New Assessment Framework aligns all assessment initiatives to UTRGV’s regional accreditation timeframe for up-to-date reporting needed for key deadlines (i.e., the fifth year interim report and reaffirmation).

The sections on this page detail the various components and expectations for assessment reporting under each framework.


JUSTIFICATION
  • Assessment Planning and Administration Management: The Assessment Frameworks will help reduce any disruptions to educational processes by increasing the flexibility for managing assessment activities.
  • Assessment Process Maturity: The Assessment Frameworks establish expectations for collecting data at multiple points in time to allow program/unit assessment processes to evolve and mature.
  • Improvement Development and Implementation: The Assessment Frameworks provide ample opportunities for programs/units to develop and implement modifications intended to impact performance.
  • Improvement Monitoring and Efficacy: The Assessment Framework enables programs/units to periodically monitor and evaluate the effectiveness of modifications over time.
  • SACSCOC Accreditation Alignment: The Assessment Framework will facilitate the preparation of the Compliance Certification Report focused on Institutional Effectiveness (C.R. 7.3, C.R. 8.2.a and C.R. 8.2.c) by aligning all assessment activities with UTRGV’s regional accreditation timeframe (i.e., the fifth year interim report and reaffirmation).

TRANSITIONAL ASSESSMENT FRAMEWORK (2015-2017)

The Transitional Assessment Framework for UTRGV includes the 2015-2016 and 2016-2017 academic years. For this two-year period, units/programs will need to identify their expected objectives; describe the means by which objectives are assessed; collect, analyze, and report on assessment data; and develop and report program/unit modifications tied to each objective. To demonstrate that unit/programs have engaged in these assessment activities, they are expected to submit the following institutional effectiveness reports to the Office of Academic and Institutional Excellence (OAIE):

CollectReporting Expectations
  • 2015-2017 Comprehensive Assessment Plan
    Identify expected outcomes and assessment methods that will be used for the duration of the reporting cycle.
    (Due: May 13, 2015)
  • 2015-2016 Annual Assessment Report
    Document assessment results/analysis/improvements from previous academic year.
    (Due: October 1, 2016)
  • 2016-2017 Annual Assessment Report
    Document assessment results/analysis/improvements from previous academic year.
    (Due: June 1, 2017)
  • 2015-2017 Continuous Improvement Review
    Provide evidence that improvements were made as a result of reviewing assessment data and making program/unit modifications during the reporting cycle.
    (Due: June 1, 2017)
assessAssessment Expectations
  • Identify multiple expected outcomes: Choose 3 to 5 expected outcomes that will be assessed during the two-year period. These outcomes should fulfill the program/unit mission and align to UTRGV’s mission.
  • Use direct assessment measures: Identify assessment methods that directly evaluate performance on expected objectives.
  • Identify expected performance standards: Establish a definition of success that will serve as a reference point for determining the extent to which objectives are achieved.
Timeline 

Download Transitional Assessment Framework (.pdf).


NEW ASSESSMENT FRAMEWORK (2017-2022)

The New Assessment Framework for UTRGV requires programs/units to identify their expected objectives; describe the means by which objectives are assessed; collect, analyze, and report on assessment data; and develop and report program/unit modifications tied to each objective. To demonstrate that unit/programs have engaged in these assessment activities, they are expected to submit the following institutional effectiveness reports to the Office of Academic and Institutional Excellence (OAIE):

CollectReporting Expectations
  • 2017-2022 Comprehensive Assessment Plan
    Identify expected outcomes and assessment methods that will be used for the duration of the reporting cycle
    (Due: March 1, 2017)
  • 2017-2018 Annual Assessment Report
    Document assessment results/analysis/improvements from previous academic year.
    (Due: October 1, 2018)
  • 2018-2019 Annual Assessment Report
    Document assessment results/analysis/improvements from previous academic year.
    (Due: October 1, 2019)
  • 2019-2020 Annual Assessment Report
    Document assessment results/analysis/improvements from previous academic year.
    (Due: October 1, 2020)
  • 2020-2021 Annual Assessment Report
    Document assessment results/analysis/improvements from previous academic year.
    (Due: October 1, 2021)
  • 2017-2022 Continuous Improvement Review
    Provide evidence that improvements were made as a result of reviewing assessment data and making program/unit modifications during the reporting cycle.
    (Due: December 1, 2021)
  • 2022-2027 Comprehensive Assessment Plan
    Identify expected outcomes and assessment methods that will be used for the duration of the next reporting cycle
    (Due: April 1, 2022)
assessAssessment Expectations
  • Identify multiple expected outcomes: Choose 3 to 5 expected outcomes that will be assessed during the four-year data collection period. These expected outcomes should fulfill the program/unit mission and align to UTRGV’s mission.
  • Collect data on at least one expected outcome annually: Programs/units will need to collect and report assessment data on at least one expected outcome every year.
  • Any expected outcome must be assessed at least twice in a four-year period. Data should be collected for any expected outcome at least two times during the four-year data collection period.
  • Use direct assessment measures: Identify assessment methods that directly evaluate performance on expected objectives.
  • Identify expected performance standards: Establish a definition of success that will serve as a reference point for determining the extent to which objectives are achieved.

Examples for planning the frequency and timing of collecting assessment data


Timeline

Download New Assessment Framework (.pdf).


Framework:
  • Why are we implementing a New Assessment Framework?

    The New Assessment Framework is designed to provide a more flexible approach to assessment practices allowing for a more meaningful process for faculty while also aligning assessment activities with UTRGV’s regional accreditation timeframe (i.e., the fifth year interim report and reaffirmation).

  • Can I keep on utilizing the previous two-year framework?

    All units/programs are required to adopt the New Assessment Framework. However, the reporting components will remain the same, as units/programs are expected to submit the following: 1) Comprehensive Assessment Plan, 2) Annual Assessment Report, and 3) Continuous Improvement Review. The major difference is that the New Assessment Framework establishes expectations over a longer period of time.

  • What is the purpose of having an assessment framework?

    UTRGV is expected to comply with SACSCOC accreditation requirements related to Institutional Effectiveness (IE). The Office of Academic and Institutional Excellence (OAIE) has established an assessment framework that allows programs and units to document their IE processes to demonstrate compliance with these requirements at every level of the institution. In sum, an assessment framework provides a clear, practical set of requirements and expectations so that programs and units can engage in the continuous improvement process. The framework also assesses the degree to which these expectations are achieved, and develops improvements using assessment data.

  • How does the New Assessment Framework coincide with external accreditation requirements for my college/school?

    Programs and units are encouraged to align their external accreditation expectations and compliance data with the New Assessment Framework. This means that, if planned carefully, your unit/program can conduct one set of assessment activities to serve both needs.

  • How is an educational degree program defined for purposes of assessment under the New Assessment Framework?

    Educational degree programs contain a coherent set of courses in a field of study that lead to a unique credential or degree awarded by the institution (BA, BBA, BS, MA, MBA, PhD, etc.). By definition, this includes program majors, stand-alone minors, and stand-alone certificates. If an educational degree program requires students to learn specific knowledge, skills, or values in a sequence of courses leading to a degree, they should have their own assessment plan.

  • Which units should engage in Unit Assessment for the New Assessment Framework?

    Units expected to engage in assessment initiatives are academic or non-academic departments/offices that have an independent role and function at the University; have an operating budget; and contribute to some operational aspect of the institution, such as administrative support services or student support services.


Planning:
  • Should my plan focus on four years or five years in the New Assessment Framework?

    Your plan for administering assessments should occur during the first four years of the framework. Under the New Assessment Framework, the first four years require programs and units to collect, analyze, and report assessment data. The fifth and final year of the New Assessment Framework is reserved for reviewing, reflecting, and planning for the next continuous improvement period.

  • Can my program/unit revise the assessment plan, even after it has been submitted to the OAIE?

    Yes. You are the owner of your assessment plan; modifications can be made as often as needed.

  • What is the minimum number of expected outcomes that should be assessed in the New Assessment Framework?

    Programs/units should identify multiple expected outcomes in their plans. The recommended practice for units and programs in doctoral degree granting institutions is to include 3 to 5 expected outcomes. At least one (1) expected outcome should be assessed on an annual basis. Also, any expected outcome included in assessment plans should be assessed twice in four years. This expectation will allow programs/unit to measure the impact of improvements made as a result of analyzing assessment data.

  • What is a curriculum map and how can it help me plan out my assessment activities for the New Assessment Framework?

    A curriculum map is a matrix or table that identifies the learning outcomes that are covered in each course in the curriculum. A curriculum map can help programs plan their assessments according to where individual learning outcomes are taught in the curriculum.

  • If my program has not yet developed a curriculum map, what should I do for the New Assessment Framework?

    After identifying the expected outcomes you plan to assess, explain – in narrative form – where in the curriculum these expected outcomes are covered and where they will be assessed.

  • Can I use the same comprehensive assessment plan for different delivery methods (on-line vs. face-to-face)?

    Programs that offer a substantial number of courses online, but are identical to the face-to-face format, can keep the same assessment plan. However, they should assess students in both the traditional and on-line learning formats to determine the degree to which they meet program expectations. If programs only offer minimal courses on-line, they do not need to disaggregate data by modality.


Assessing/Collecting Results:
  • How often do expected outcomes need to be assessed for the New Assessment Framework?

    Any expected outcome included in your assessment plan should be assessed in at least two distinct academic years over a four year period. Collecting data at two distinct points in time will allow units/programs to measure the impact of any improvements made as a result of analyzing assessment data.

  • If I assess twice before proposing improvements, do I meet the New Assessment framework’s requirements?

    No. Each expected outcome must be assessed a minimum of two times over a four year period. The first assessment point will occur before proposing improvements; the second assessment point will occur after improvements have been implemented. The second assessment point helps to assess the impact of improvements.

  • Can I utilize the same measure(s) for assessing a particular expected outcome at two different points in time throughout the New Assessment Framework?

    Yes. Using the same measure(s) at two distinct points in time will allow comparisons to be made; therefore, it is expected that the measure(s) remain constant.


Reporting Results/Proposing Improvements:
  • Will the New Assessment Framework expect programs/units to propose or implement improvements?

    Yes. Proposing and implementing changes are elements that are part of the assessment cycle and, thus, required in the New Assessment Framework. Although programs/units can propose and implement improvements at any point in time, the OAIE will ask them to report these actions on a yearly basis in the Annual Assessment Report and at the end of the cycle in the Continuous Improvement Review.

  • Do I have to wait until the end of an academic year to report results and propose improvements under the New Assessment Framework?

    Yes. The OAIE will ask units/programs to reflect on the previous academic year and report assessment results by October 1 on an annual basis. However, units/programs can analyze assessment data and develop improvements at any time during the New Assessment Framework.

  • Am I allowed to change an expected outcome during the New Assessment Framework timeline?

    Yes. However, units/programs must be mindful of assessing the expected outcomes at least twice before the end of the data collection years in the New Assessment Framework.

  • How should my program/unit prepare for reporting results under the New Assessment Framework? What types of information would we be expected to provide?

    The information provided by each program will vary and will be dependent on the measures utilized. However, all programs should provide a summary of their assessment results via a table, graph, or chart and an accompanying analysis of this information.

  • What are acceptable methods and measures that my program/unit should consider using under the New Assessment Framework?

    Educational programs should primarily rely on evaluating student work (i.e., use direct assessment measures) to determine the extent to which students achieve expected outcomes. They are encouraged to use indirect measures to evaluate student attitudes and values related to expected learning outcomes. Units should also primarily rely on direct measures. However, unlike educational programs, units should focus on using measures to evaluate the operational aspects of a department or office (accuracy, satisfaction, efficiency, productivity, effectiveness). This includes anything that can be observed (i.e., budget reports, database trends, peer comparisons, institutionally managed data, other data). Satisfaction evaluations are highly encouraged measures of attaining expected outcomes for units.

  • Will the New Assessment Framework require units/programs to provide evidence of decision-making on assessment initiatives, such as planning, analyzing data, and making improvements?

    Yes. Units/programs should provide meeting minutes, agendas, and other evidence to demonstrate that assessment initiatives occurred.