|
|
 |
 |
Results by Question
1.
What advice
can you provide regarding how to conduct a successful ABET review (e.g.,
best practices you can share)?
|
Comment |
# |
|
1.
Be sure you can demonstrate that a continuous improvement process is in
place and that changes have been made as a result of assessment |
9 |
|
2.
Establish an ABET committee with representation from each department
for the purpose of coordinating and sharing best practices; meet
regularly |
8 |
|
3.
Collect minimal information to get maximum results – be sure it is
sustainable; effective and not burdensome |
5 |
|
4.
Develop clearly stated goals and objectives, mission statements, etc. |
4 |
|
5.
Have good documentation, which gives the evaluator a good feel for what
is going on in the program (minutes, memos, etc.) |
4 |
|
6.
Critique each others work/hold mock review |
4 |
|
7.
Start early |
3 |
|
8.
Get faculty buy-in (if possible early in the process) |
3 |
|
9.
Use multiple assessment tools, don’t rely on too few measures,
preferably 2 measures for each outcome |
3 |
|
10.
Be sure to develop primary sources of data and don’t just rely on
secondary sources (e.g., surveys) |
3 |
|
11.
Align your outcomes closely with ABET 3a-k and provide a mapping for
ABET reviewers, but o.k. to put in your own words |
3 |
|
12.
Be sure your self-report is well-done and speaks for itself |
3 |
|
13.
Require annual review of assessment progress at degree program level |
3 |
|
14.
We encourage faculty to become ABET evaluators |
3 |
|
15.
Move in the direction of course based assessment and don’t rely too
much on surveys |
2 |
|
16.
Make it simple for the reviewers – don’t overwhelm with too much
information and organize well |
2 |
|
17.
Use a flowchart to show entire assessment process |
2 |
|
18.
Use Online Assessment Tracking System (OATS) or other website on which
every program must demonstrate learning objectives/outcomes, methods
for assessment, data collected, actions taken, feedback loop closed,
etc. |
2 |
|
19.
Hired a staff person who helped with the assessment process and
workshops and the compilation of survey data |
1 |
|
20.
Review previous ABET reports to see where problems were in the past |
1 |
|
21.
Collect student work and use the richer pieces (e.g., student projects
and senior design reports) to get several types of information |
1 |
|
22.
Develop a mapping that shows how your course outcomes match to your
program outcomes. |
1 |
|
23.
Don’t start taking measures until you are sure what you want to measure
and know that you will use it |
1 |
|
24.
Hold an assessment fair at which faculty can be exposed to and trained
on different assessment tools. |
1 |
|
25.
Conduct longitudinal studies and track a cohort of students across
entering freshmen, mid-curriculum, exiting seniors. |
1 |
|
26.
Do a good job on the first day of the visit – you may not even need a
second day |
1 |
|
27.
Use standard templates whenever possible |
1 |
|
28.
Make sure that all course descriptions have learning outcomes and that
you have a mechanism to measure these outcomes. Make faculty
responsible for their own course assessments and ensure they address
identified weaknesses. |
1 |
|
29.
Conduct interviews of small groups of students and the junior and
senior level |
1 |
|
30.
Having faculty members fill out a self-evaluation form at the end of
every semester |
1 |
|
31.
Prepare faculty, students and staff in advance of the ABET program
evaluators’ visit |
1 |
|
32.
If you say you are going to do something, be sure to do it! ABET will
check to make sure that you did what you said you would do |
1 |
|
33.
Focus the college wide data, help the programs to focus on their
self-studies and provide a framework for them to conduct their
assessments |
1 |
|
34.
Strong industrial advisory boards is a critical ingredient at the
department level |
1 |
|
35.
Invite knowledgeable experts to come talk with faculty about ABET
matters – typically people who understand assessment very well |
1 |
|
36.
Get in touch with your Program Evaluators early and be responsive in
providing them with any material they need |
1 |
|
37.
No advice to offer – we are at the beginning of this new scheme for
doing things and what we have mostly done so far is promise what we
will do in the future |
1 |
2.
How
successful have you been at ___________________ (institution)?
|
Comment |
# |
|
1.
Extremely successful – all programs have received Next General Review |
4 |
|
2.
Very successful – nearly all of our programs have received Next General
Reviews |
12 |
|
3.
Reasonably successful – a number of our programs have received Next
General Review |
2 |
|
4.
Mixed review – several of our programs made Next General Review, but
several had interim reports required |
3 |
|
5.
Not successful – have had multiple interim visits and interim reports
and very few programs that have made Next General Review on their
first review |
1 |
|
6.
First EC2000 review is coming up in next year or so, so unsure of
success with new criteria yet |
4 |
|
7.
Unable to be/refused to be specific |
1 |
3.
What is the
role of the College vs. the role of the programs with respect to ABET
responsibilities?
|
Comment |
# |
|
1.
College
coordinated team of ABET coordinators which met frequently to share
best practices |
17 |
|
2.
College
coordinated and/or administered various surveys (generation, mailing,
collection and analysis) including alumni, senior, employer |
15 |
|
3.
College
completed the Appendix II document (common material at college level) |
10 |
|
4.
Programs have to be the primary responsible party, because they are
what is being evaluated, not the college – most work was done at the
program level |
8 |
|
5.
College
gathers data from and/or assesses core courses and general education
courses (math, physics, chemistry, etc.) |
6 |
|
6.
College
attempted to figure out what level of detail is required to assess
3a-k outcomes |
2 |
|
7.
College
worked one-on-one with departments/programs to make sure they were
ready for the visit |
2 |
|
8.
College
coordinated with Career Services to obtain data re: interns, co-ops,
and career placement |
2 |
|
9.
College
enforces some standards with respect to common documentation/templates
(faculty resumes, courses syllabi, etc.) |
2 |
|
10.
College
helped the programs to get organized/coordinated |
2 |
|
11.
Office
at institute level performs central assessment and analysis |
2 |
|
12.
College
coordinated team of staff members who do ABET analysis work, which met
frequently to share best practices |
1 |
|
13.
College
does quality control on self-studies |
1 |
|
14.
College
coordinates overall ABET visit |
1 |
|
15.
College
computing services group will be implementing an automated prompting,
scheduling and tracking tool for assessment of course outcomes |
1 |
|
16.
College
is developing software that takes our course grades data base and
computes a GPA for 3a-k based on the percent of course that is
involved in each criterion and the GPAs that were obtained – but may
not fly with ABET since grades aren’t always deemed to be a good
indicator |
1 |
|
17.
College
oversees co-op survey of employers |
1 |
|
18.
College
assesses cross-department programs that don’t have a home |
1 |
|
19.
College
conducted longitudinal studies |
1 |
|
20.
College
analyzed FE exam data on behalf of the departments/programs and
distributed results |
1 |
|
21.
College
established intermediate deadlines at the college level which every
program needed to meet |
1 |
|
22.
College
has an educational psychologist who makes sure that departments are
gathering information they really want and will really use and that
action takes place as a result of information gathered |
1 |
|
23.
College
provided some funding to pilot new assessment tools |
1 |
|
24.
Believe
that the College should have centralized more than they did |
1 |
4.
Do you
employ a consistent approach to ABET across your college, or do you let
programs do things their own way?
|
Comment |
# |
|
1.
No, we
let the departments and programs do whatever they wanted to do – they
are the best judge of what is needed (most shared information about
their best practices with other programs) |
20 |
|
2.
Yes, we
employed a systematic approach, but allowed for some customization
within the departments or programs |
3 |
|
3.
Yes, we
employed a systematic approach across the college and encourage
everyone to do many things in a similar way; programs populate
templates with data |
1 |
|
4.
Used a
common template for the self-studies |
3 |
|
5.
Moving
in the direction of more consistency with the next ABET cycle |
2 |
|
6.
All
programs were required to meet specific deadlines |
1 |
|
7.
Used
consistent vocabulary with a glossary that explained terms |
1 |
|
8.
College
provided a common philosophy – do what makes sense for your program
and think secondarily about what ABET will want |
1 |
5.
There seems
to have been some confusion in the past with respect to definitions of
objectives and outcomes (and a few other words). While I think I have a
grasp of the difference, what insights can you share re: this topic and
how you have distinguished them?
|
Comment |
# |
|
1.
Outcomes are what the student can do at graduation; objectives are
what the students can accomplish several years later |
9 |
|
2.
Outcomes are near-term; objectives are long-term |
2 |
|
3.
Outcomes are what you can measure as students leave and up to 3 years
out. Objectives are at a higher level, are more difficult to measure
and are what we want our graduates to be known for collectively.
Objectives are more likely to show up a few years out. |
2 |
|
4.
Outcomes are what we measured students on; objectives are high-level.
If student met outcomes, by definition they met objectives |
1 |
|
5.
Outcomes are skills that students have when they leave here.
Objectives are more general and difficult to measure |
1 |
|
6.
Objectives are what the faculty wants to teach; outcomes are what we
expect the students to learn |
1 |
|
7.
Program
outcomes are more like goals and unmeasureable as such; you need to
figure out how to measure them, which may require breaking them down
into component pieces |
1 |
|
8.
Outcomes are all campus-based, and based on what the students can do
when they graduate; objectives are practice-based, and based on what
the graduates can do after they have begun to practice the profession |
1 |
|
9.
Outcomes (3a-k) are a body of knowledge defined by industry,
professional societies, etc. Program criteria (8) are defined by
professional organizations. Objectives (2) are defined by programs
considering what is important to their program and in their
environment |
1 |
|
10.
Outcomes are what a recruiter would use to assess your students skills
when making a hiring decision; objectives are what a recruiter would
use to tell whether they could use your students in their organization |
1 |
|
11.
Very
confusing – our definitions don’t match ABET’s definitions, but we are
sticking with them because we have been measuring against them for
several years |
1 |
|
12.
Don’t
think new ABET definition of objectives is appropriate – still trying
to figure it out. It is not our role as academics to guarantee what
our students will do down the road or what they will accomplish in
their career – perhaps we are reading it too restrictively and it
should be interpreted as the difference between measuring potential
vs. what the students have actually done with their education |
1 |
|
13.
This is
a weakness of ABET because it has not been well-communicated |
1 |
|
14.
ABET is now providing better documentation on this topic |
1 |
|
15.
Never thought about it |
1 |
|
16.
No relevant response to this question |
1 |
|
17.
Don’t ask me because ABET has changed their approach to this since we
went through our last review. |
1 |
|
18.
Still confused on this matter/Don’t know/Can’t recall difference |
4 |
6.
What have
you found to be your most successful measures for criterion number 2
(objectives)?
|
Comment |
# |
|
1.
Going
to industry for their feedback – mainly through the advisory boards |
20 |
|
2.
Use
alumni surveys |
18 |
|
3.
Used
surveys with industry/employers (with varying degrees of success –
those who were successful attributed this to good established
relationships with industry, several said this was not an effective
measure as it was too difficult to get good data; one even called it
an abysmal failure!) |
9 |
|
4.
Gather
data or conduct focus groups with key constituents (students, alumni,
employers, faculty, etc.) |
4 |
|
5.
We
don’t measure objectives directly. We map department outcomes to
department objectives – if the students have met the outcomes, we have
met the objectives |
4 |
|
6.
Job
placement information |
3 |
|
7.
Graduate school track record |
3 |
|
8.
Obtain
information from recruiters |
2 |
|
9.
Be sure
to have objectives listed in the catalog and on the website |
2 |
|
10.
Number
of students who have obtained their PE |
1 |
|
11.
Use
Career Services (or some similar type of group) to collect information
about where students are going |
1 |
|
12.
Invite
faculty to provide feedback, suggesting changes to objectives |
1 |
|
13.
We
chose our objectives in line with the strength of the program;
objectives were chosen to fit what we were already doing |
1 |
|
14.
Use
course evaluations |
1 |
|
15.
Can be
done on a longer time frame and not evaluated every year |
1 |
|
16.
We keep
getting information from ABET on what won’t work to measure this, but
insufficient guidance on what will work! |
1 |
|
17.
Be sure
to define your constituents first – who will care? |
1 |
7.
What have
you found to be your most successful measures for the 3a-k outcome
criteria?
|
Comment |
# |
|
1.
Course
work examples (carefully displayed and collected at strategic points
during their studies) |
11 |
|
2.
Use
embedded course-based measures such as online simulations, test
questions, lab reports, etc. |
7 |
|
3.
FE exam
data |
7 |
|
4.
Senior
surveys |
7 |
|
5.
Surveys
with students |
5 |
|
6.
Capstone/senior design courses/projects (some have industry
evaluators, too) |
5 |
|
7.
Portfolios (mainly for smaller schools, smaller programs, or where
portfolios make sense for the students – e.g., Graphic Design) |
4 |
|
8.
Align
your outcomes closely with ABET 3a-k and provide a mapping for ABET
reviewers |
4 |
|
9.
Alumni
surveys |
3 |
|
10.
Obtained information from instructors re: what their courses covered
and how well their course met certain criteria |
3 |
|
11.
Obtained information from instructors re: preparation of students in
specific areas in which we are interested. |
2 |
|
12.
Plot
importance of outcome to student on one axis of graph and plot how
well program did on other access – window of opportunity lies where it
is high importance, but didn’t perform well |
2 |
|
13.
Faculty
are performing the measurements regularly and as an expected part of
their work |
2 |
|
14.
Believe
course grades are a legitimate way to measure, which ABET doesn’t
necessarily agree with |
2 |
|
15.
Surveyed our industrial advisory board/gathered information from
employers |
2 |
|
16.
Employer surveys |
1 |
|
17.
GRE
exam results and admission to grad school at respected institutions |
1 |
|
18.
Information from placement center re: # of interviews students have
had, positions taken, salaries offered |
1 |
|
19.
Be sure
to have at least two assessment techniques for each outcome, one
direct and one indirect |
1 |
|
20.
We
measured course outcomes and then mapped them to program outcomes and
then to objectives |
1 |
|
21.
Interviews with juniors and/or seniors |
1 |
|
22.
Still
hashing this out – no agreement reached yet |
1 |
8.
How do you
ensure that you are meeting the “softer criteria” such as life-long
learning, ethics, etc.? What measures do you use?
General responses:
|
Comment |
# |
|
1.
Use the
capstone/senior design project to demonstrate some of these items |
7 |
|
2.
We ask
students/interview students to see if they understand this information |
6 |
|
3.
We ask
students/interview students to see if they believe they are getting
sufficient exposure to these topics and understand them |
6 |
|
4.
We have
various courses that seek to teach these softer criteria; collect
course work from these (e.g., intro to Engineering, other courses) |
5 |
|
5.
Use
general education courses to demonstrate some of these items |
4 |
|
6.
Rely on
alumni surveys |
4 |
|
7.
Discussions with faculty reveal where these softer criteria are being
addressed in their coursework – be sure they understand they are
responsible for certain outcomes |
3 |
|
8.
Made it
clear to ABET in which courses we taught these topics (use a matrix to
show courses against criteria) |
3 |
|
9.
Our
weakest point/little to show in way of hard results |
3 |
|
10.
Must
start by defining what you mean by these softer criteria for a
particular program |
2 |
|
11.
Rely on
feedback from our employers |
2 |
|
12.
Rely on
feedback from co-op and/or internship program |
2 |
|
13.
We
conduct focus groups to gather information on the softer criteria |
1 |
|
14.
We have
seminar courses that invite external speakers to speak to the students
about these subjects – we can show exposure |
1 |
|
15.
The
real key here is to thoughtfully develop the measures for these
criteria – what does it take to be a life-long learner? What are the
contemporary issues for this program? |
1 |
|
16.
You can
merge the measurements to allow for easier evaluation – have students
practice oral presentations while talking about ethics |
1 |
|
17.
Our
EPICS (Engineering Projects in Community Service) program helps us
demonstrate some of this |
1 |
|
18.
We do
team project checks in some of our courses to measure how well
students are meeting softer criteria |
1 |
|
19.
When we
prioritized outcomes, these were prioritized on the low end;
therefore, we don’t measure them specifically |
1 |
Life-long learning specific responses:
|
Comment |
# |
|
1.
We determine if students/alumni use the internet for research or can
conduct self-directed learning (e.g., through library) |
4 |
|
2.
We ask if alumni have pursued additional degrees or certifications |
4 |
|
3.
We ask if alumni take additional classes (e.g., continuing ed) or
in-house training |
4 |
|
4.
We ask if students/alumni subscribe to publications/journals/newspaper |
3 |
|
5.
We ask if students/alumni are members of any professional group |
2 |
|
6.
W ask if students/alumni are publishing/working with a professor |
1 |
|
7.
We ask if students/alumni participate in co-ops or internships |
1 |
|
8.
We ask if students/alumni attend conferences |
1 |
|
9.
We ask if they have tutored an engineering student |
1 |
|
10.
Teach life-long learning in undergrad engineering seminar required for
all students graduating within the next year. Students required to
develop a life-long learning plan as part of course – evaluate results |
1 |
|
11.
Ensure that students have been exposed to the concept (understand need
to participate in professional societies, attend conferences, pursue
additional education). |
1 |
Ethics specific responses:
|
Comment |
# |
|
1.
We have a required course in ethics – (one respondent mentioned that it
is co-taught between engineering and Department of Philosophy) |
2 |
|
2.
Ethics component on FE exam |
1 |
|
3.
Summer ethics workshop for faculty about how to incorporate ethics into
their courses – about 1/3 of faculty have attended, are paid for 1
week of pay to attend |
1 |
9.
Do you
require your students to take the FE exam? If so, do you pay for part of
the costs of the exam or compensate them in some other fashion?
|
Comment |
# |
|
1.
Yes, it
is required and we compensate them |
0 |
|
2.
Yes, it
is required by one or more of our departments and they do not
compensate them |
2 |
|
3.
Not
required, but definitely encouraged and we do compensate students who
pass it (not just take it) |
2 |
|
4.
Not
required, but definitely encouraged; we do not compensate
students |
5 |
|
5.
Not
required, and we do not compensate them |
10 |
|
6.
Not
required, not sure if compensated |
5 |
|
7.
Becoming a PE is enough of an inducement for the Civil students |
3 |
|
8.
We hold
review sessions/tutorials to help prepare students |
3 |
|
9.
We hold
information session to inform students about FE |
1 |
|
10.
We do a
prep test for the FE exam which students can take for free |
1 |
|
11.
We
would help the student pay if finances were an issue for him/her |
1 |
|
12.
Depends
on the department |
1 |
|
13.
Don’t
know/not certain |
2 |
10.
Do you use
portfolios – with what success?
|
Comment |
# |
|
1.
No, we
don’t use portfolios (too much work to manage) |
15 |
|
2.
Started
to use them, but too much work, don’t use them now |
2 |
|
3.
One
program uses portfolios because the faculty does such a poor job of
gathering the necessary data |
1 |
|
4.
Some
programs/departments used some portfolios, but not used in any
consistent sense and not attempt to gather data longitudinally |
1 |
|
5.
Some
programs/departments use portfolios if it makes sense for them to do
so (especially if students can benefit from doing so – such as
computer or graphic art or if it is a very small program) |
6 |
|
6.
Yes, we
use portfolios in many of our programs/departments |
1 |
|
7.
Yes, we
use portfolios in all of our programs/departments |
3 |
|
8.
We
collect work from faculty |
1 |
|
9.
We
collect work from students |
1 |
|
10.
Student
must agree to have work collected for them |
1 |
11.
Do you use
Senior Surveys? Would you be willing to share a sample?
|
Comment |
# |
|
1.
Yes, we
use senior surveys – unwilling, unable, or did not provide a copy |
16 |
|
2.
Yes, we
use senior surveys and I am happy to provide a copy or reference you
to our website to pull a copy |
11 |
|
3.
No, we
had low response rates, so now we interview a random sample of our
graduating seniors |
1 |
|
4.
Our
surveys were web-based |
7 |
|
5.
Students must “check out” through their department to graduate
(referred to variously as a senior audit or degree audit) and we
require they fill out a survey at that time |
5 |
|
6.
We also
do exit interviews with students |
5 |
|
7.
Students were offered a some sort of incentive (e.g., school
memorabilia, graduation invitations, etc.) for completing the survey |
5 |
|
8.
Use EBI
surveys, administered in classes |
3 |
|
9.
Used to
use EBI, moved in direction of in-house university survey |
2 |
|
10.
We also
use surveys at freshmen level |
2 |
|
11.
We
reminded students to complete the form, but it was not required |
2 |
|
12.
Survey
was a required assignment in a senior capstone or senior seminar
course |
2 |
|
13.
Response rate was good |
2 |
|
14.
Surveys
conducted in a classroom had the best response |
1 |
|
15.
Voluntary, but students asked to complete survey when they pick up
their caps and gowns |
1 |
|
16.
We also
use surveys at mid-curriculum level |
1 |
|
17.
Our
surveys are course based surveys – one for every course because we
wish to identify weaknesses in course outcomes |
1 |
12.
Do you use
Alumni Surveys? Would you be willing to share a sample? How often do you
send surveys? What has been your response rate? Do you use any
inducements (e.g., gift certificates or money) to improve the response
rate?
|
Comment |
# |
|
1.
Yes, we
use alumni surveys – unwilling, unable, or did not provide a copy |
15 |
|
2.
Yes, we
use alumni surveys and I am happy to provide a copy or reference you
to our website to pull a copy |
11 |
|
3.
No, not
yet – too few alumni, but will use in the future |
1 |
|
4.
Recommend 1 and 3 year window out |
2 |
|
5.
Recommend a 1 year window out |
1 |
|
6.
Recommend a 2 year window out |
1 |
|
7.
Recommend a 3-5 year window out |
1 |
|
8.
Recommend a 1,2,3,5 year window out |
1 |
|
9.
Select
3 students from each of the last 5 years for a total of 15 being
interviewed each year |
1 |
|
10.
We
survey a group all at once (e.g., graduates from 1997-2003) |
1 |
|
11.
Response rate of less than 10% (or very low) with no inducements |
3 |
|
12.
Response rate placed between 10-30% with no inducements |
11 |
|
13.
Response rate placed between 30-45% with no inducements |
3 |
|
14.
Response rate of 35% with an inducement of a drawing for football
tickets |
1 |
|
15.
Response rate is nearly 60% with no inducements beyond stressing
importance (to the college and the value of their degree) of
completing and returning survey |
1 |
|
16.
Response rate has changed over time – 23% first year, 19% second year
(return envelope and stamp sent), 25% (drawing for football tickets),
30% (when we went to web-based) |
1 |
|
17.
Not
sure of response rate – but no inducements were used |
2 |
|
18.
One
department tried sending a crisp one dollar bill, not sure with what
success |
1 |
|
19.
May
begin using inducements in the future to boost response rate (e.g.,
tickets to football game) |
1 |
|
20.
Finding
the right faculty member to send the alumni survey is key |
1 |
|
21.
Tell
seniors that they will see this survey in a few years and why it is
important to return it |
1 |
|
22.
We
don’t conduct this survey every year |
2 |
|
23.
Web-based surveys were used |
5 |
|
24.
Alumni
survey is sent out at campus level |
5 |
|
25.
Use EBI
data, mainly at the college level |
2 |
|
26.
Alumni
advisory board is being used to gather this information |
1 |
13.
What role
do your Department Advisory Boards play in your ABET process?
|
Comment |
# |
|
1.
Provide
feedback on/help refine program objectives |
19 |
|
2.
Provide
feedback on/help refine program outcomes |
8 |
|
3.
They
provide feedback on our curriculum, advise on curriculum changes |
7 |
|
4.
They
review our assessment data |
6 |
|
5.
Meet
with students in small groups to gather data about the program |
4 |
|
6.
Attend
some/all of our senior design/projects presentations and provide
feedback or help grade |
4 |
|
7.
Provide
feedback on what graduates need to know to be prepared for jobs with
their companies |
3 |
|
8.
Asked
for feedback on self-study |
2 |
|
9.
Advisory Board is required to write a report about their findings and
this goes to the Dean and Department Chair |
2 |
|
10.
Not
much of a role at this time |
2 |
|
11.
Varies
by department/program – can’t provide specifics |
2 |
|
12.
They
suggest actions which we might pursue to address issues that show up
in assessment data |
1 |
|
13.
They
provide a reality check – e.g., told us not to bother with employer
surveys as we wouldn’t get good participation because there was
nothing in it for the employers |
1 |
|
14.
Periodically they attend student classes |
1 |
|
15.
Subset
of advisory board came in and held a mock review |
1 |
|
16.
Helpful
at a global level, not so helpful with the details |
1 |
|
17.
Sometimes involved in various continuous improvement processes |
1 |
|
18.
Include
some advisory board members at the Monday lunch during the ABET visit |
1 |
|
19.
Ask for
feedback from our advisory board about how our graduates compare with
graduates from other schools, but they are better at telling us what
skills they are looking for |
1 |
14.
How do you
ensure that the things you learn are incorporated into the feedback loop
and that action is taken to correct problems?
|
Comment |
# |
|
1.
Use
curriculum committee or undergrad committee to take action; keep
minutes |
13 |
|
2.
Require
an annual (or periodic) report with an assessment component, which
provides a centralized archive of information re: assessment |
9 |
|
3.
Assessment committee/ABET coordinator looks at the data that is
collected and decides what action to take. |
8 |
|
4.
Departments/programs are provided with data that is collected
centrally and they can decide what actions to take – allows for
comparison with other departments/programs |
3 |
|
5.
Fear
this will be an issue with next ABET review; most difficult area |
3 |
|
6.
Keep
minutes of meetings and track outcomes |
2 |
|
7.
Items
which need attention are discussed with our advisory board |
2 |
|
8.
Items
which need attention are discussed at the faculty retreat |
2 |
|
9.
Keep
good records; documenting decisions made and follow-up |
2 |
|
10.
Maintain an annual log, documenting program outcomes, how assessed,
what we discovered and action we took to address any problems that
were identified. Later we measure what subsequent information we
gather to confirm the decisions we made |
2 |
|
11.
Can’t
ensure – you just use leadership to guide the effort/have good
intentions |
2 |
|
12.
We show
we gave consideration to all feedback received – but we don’t take all
feedback, as it may be biased and not in the long term best interests
of the students |
1 |
|
13.
Annual
program review meetings within each department (could be within
curriculum committee or ABET coordinating committee) |
1 |
|
14.
Annual
college-level discussion of ABET at Undergrad Studies committee |
1 |
|
15.
Faculty
assess courses, either individually or in groups, in combination with
curriculum committee review |
1 |
|
16.
Require
faculty to note on course approval forms which ABET criteria are being
addressed by the course |
1 |
|
17.
Higher
level courses can provide feedback to lower level courses about how
well prepared students are to take their courses |
1 |
|
18.
Advisory Board reports provide insights and may result in additional
work being done to address issues that are raised |
1 |
|
19.
We have
outside visitors that evaluate our preparation and point out if we are
not using a feedback loop to incorporated the knowledge gained from
the assessment process |
1 |
|
20.
Require
action plans as a result of identified problems |
1 |
|
21.
There
must be a systematic process in place; anecdotal evidence will not be
deemed sufficient by ABET |
1 |
|
22.
This is
an issue I am trying to get our departments/programs to focus on |
1 |
15.
How do you
engage your faculty (not to mention Department Chairs) to ensure they
understand the importance of ABET?
|
Comment |
# |
|
1.
The
Dean, Associate/Assistant Deans led in the ABET process and are
supportive and active participants and provide top down direction and
leadership |
10 |
|
2.
Engage
at faculty meetings (but not well-attended in all cases) |
6 |
|
3.
Faculty
retreats |
5 |
|
4.
Training sessions/workshops with faculty |
3 |
|
5.
Get
Department Chairs on board and then the Department Chairs can take the
lead with their faculty and set the tone |
3 |
|
6.
The
assessment committee is one way to engage faculty |
3 |
|
7.
The
assessment committee communicates assessment results to the entire
faculty |
2 |
|
8.
Mock
site visits were used |
2 |
|
9.
Provided faculty with a limited amount of information they were asked
to read |
2 |
|
10.
Faculty
were extensively involved in developing outcomes and objectives |
2 |
|
11.
The
repetition method: Send lots of email to faculty/tell them over and
over again |
2 |
|
12.
ABET
coordinating committee is a subset of the Undergrad Studies committee |
1 |
|
13.
Work to
involve those faculty who are interested and understand the value of
assessment |
1 |
|
14.
Send
faculty to ABET workshops and have them come back and train others |
1 |
|
15.
Faculty
get an assessment update at least once/semester |
1 |
|
16.
Faculty
are engaged in writing course descriptions in a common format,
providing vitae, etc. |
1 |
|
17.
Faculty
are involved in providing materials for the portfolio |
1 |
|
18.
Provide
encouragement and enforce deadlines |
1 |
|
19.
We are
a newer faculty and created program with ABET in mind, so it has not
been difficult to engage the faculty |
1 |
|
20.
One-on-one coaching |
1 |
|
21.
We win
the faculty over by showing we are addressing specific problems in the
curriculum (e.g., students not well prepared in math) |
1 |
|
22.
Do
playful things to engage faculty – like give lollipops for those who
get their assessment information completed on time |
1 |
|
23.
We are
useful to faculty who want us to help them acquire information they
need for writing proposals and we have a fairly large assessment
effort underway |
1 |
|
24.
We win
the faculty over with innovative things we do in the curriculum |
1 |
|
25.
It is
difficult to maintain the momentum between visits |
1 |
|
26.
We
pointed to past failures (interim visits and interim reports), which
was a way to demonstrate how bad it could be! |
1 |
|
27.
Didn’t
try to involve faculty; sought to minimize their involvement and keep
them focused on research and teaching |
1 |
|
28.
Tried
newsletters; but they didn’t work |
1 |
|
29.
Everyone understands the importance, but nobody wants to do anything
about it |
1 |
16.
Do you have
“ABET Coordinators” for each program? How do you reward them for their efforts
related to ABET (e.g., summer pay, course forgiveness, credit for service
component of evaluation)?
|
Comment |
# |
|
1.
Yes, we
have ABET coordinators and their compensation depends upon the
department/program (some yes/some no) |
12 |
|
2.
Yes, we
have ABET coordinators, but they tend to do this work as part of a
larger role for which they are compensated in one way or another
(e.g., Associate Chair, Department Chair) |
6 |
|
3.
Yes, we
have ABET coordinators and they receive course relief or course
forgiveness |
4 |
|
4.
Yes, we
have ABET coordinators but they do not receive compensation |
4 |
|
5.
Yes, we
have ABET coordinators, but their main compensation is goodwill,
appreciation and/or lunch |
3 |
|
6.
Yes, we
have ABET coordinators and they receive summer pay |
2 |
|
7.
Yes, we
have ABET coordinators and they are provided with additional
administrative/staff support |
2 |
|
8.
Yes, we
have ABET coordinators and I am not sure if they are compensated |
2 |
|
9.
No, we
do not have ABET coordinators |
0 |
|
10.
May be
assigned in lieu of bringing in research contracts |
2 |
|
11.
Suggest
using older tenured faculty whose research efforts are winding down –
may have more time to give to this |
1 |
|
12.
It is
better to use tenured faculty in this role, so they are not worried
about the effect of their participation on tenure decisions |
1 |
17.
What other
bodies accredit your institution? Do you see linkages between ABET and
this other accrediting body?
|
Comment |
# |
|
1.
North
Central; no linkages |
5 |
|
2.
North
Central; used some ABET information for the North Central effort |
4 |
|
3.
North
Central; used some North Central information for ABET review |
2 |
|
4.
Middle
States; no linkages |
1 |
|
5.
Middle
States; used some Middle State information for ABET review |
1 |
|
6.
Middle
State; no direct linkages, but Middle States is becoming more focused
on assessment/best practices than ever before |
1 |
|
7.
Southern Association of Colleges and Schools; seek to make linkages
where possible, what we produce for one can be used for the other |
3 |
|
8.
Southern Association of Colleges and Schools; SACS is now focused more
on outcomes assessment so will provide opportunity for more linkages
in the future |
2 |
|
9.
Southern Association of Colleges and Schools; linkage info not
provided |
1 |
|
10.
Western
Association of Schools and Colleges; no direct linkages |
1 |
|
11.
Western
Association of Schools and Colleges; moving towards assessment based
accreditation so there is starting to be more commonality |
1 |
|
12.
Accreditation of Western Universities; no direct linkages |
1 |
|
13.
Northwest Association of Universities and Colleges; we do attempt to
coordinate between the two; use ABET data for Northwest |
1 |
|
14.
Some
schools have successfully made this connection |
1 |
|
15.
Not
sure about the answer to this question |
2 |
18.
Are there
particular articles that you would suggest I read?
|
Comment |
# |
|
1.
Articles by Gloria Rogers |
4 |
|
2.
Check
out NSF website and their sponsored coalitions (Foundation, Succeed,
Greenfield, etc.) |
3 |
|
3.
Journal
of Engineering Education |
3 |
|
4.
ABET
published material such as criteria document and Policies and
Procedures Manual or their website |
3 |
|
5.
Articles by Daina Briedis (Michigan State) |
2 |
|
6.
Articles or book by Trudy Banta (IUPUI) Building a Scholarship of
Assessment |
2 |
|
7.
Rose-Hulman
best practices documents/proceedings |
2 |
|
8.
JEE
92:1, page 7-25, year 2003, “Designing and Teaching Courses to Satisfy
the ABET Engineering Criteria” by Richard Felder and Rebecca Brent |
1 |
|
9.
Check
out the ASEE regional conference on line |
1 |
|
10.
Barbara
Olds and Ron Miller (Colorado School of Mines) have written several
articles |
1 |
|
11.
International Journal of Engineering Education (Volume 18, Issue 2 in
2002) |
1 |
|
12.
The
Green Paper (Aerospace) and the White Paper |
1 |
|
13.
Book by
James O. Nichols, “The Departmental Guide and Record Book for Student
Outcomes Assessment and Institutional Effectiveness” |
1 |
|
14.
Accreditation of Engineering Programs, A NEEDHA White paper |
1 |
|
15.
Criteria 2000: Lessons Learned, by C.D. Avers |
1 |
|
16.
Proven
Skills: The New Yardstick for Schools”, IEEE Spectrum, September 2000 |
1 |
|
17.
Liberal
Education Division of ASEE put out a white paper on “Recommendations
for liberal education in engineering” – presented at ASEE in June 2002 |
1 |
|
18.
Check
the web |
1 |
|
19.
Nothing
comes to mind |
6 |
19.
Are there
particular conferences that you would suggest I attend?
|
Comment |
# |
|
1.
Rose-Hulman
conference |
13 |
|
2.
ASEE
conference and workshops |
10 |
|
3.
ABET
annual conference in October |
9 |
|
4.
ABET
Dean’s Day in July before review |
5 |
|
5.
ABET
workshops |
4 |
|
6.
FIE
conference |
4 |
|
7.
ABET
Program Evaluator training |
3 |
|
8.
AAHE
conference |
2 |
|
9.
Professional society workshops (e.g., NEEDHA, now ECEDHA) |
2 |
|
10.
SIE
conference (FIE?) |
1 |
|
11.
NSF
sponsored conferences |
1 |
|
12.
Assessment Institute at IUPUI in November |
1 |
|
13.
Trudy
Banta’s conference |
1 |
|
14.
Anything by Daina Breidis |
1 |
|
15.
Association for Institutional Research national conference –
assessment track – www.airweb.org |
1 |
|
16.
Haven’t
attended any conferences in this area |
2 |
20.
With whom
else would you suggest I speak?
|
Comment (Bold
indicates interviewed in this survey) |
# |
|
1.
Gloria
Rogers, Rose-Hulman |
7 |
|
2.
Barbara
Olds, Colorado School of Mines (in DC right now) |
4 |
|
3.
Ron Miller,
Colorado
School of Mines |
2 |
|
4.
Trudy
Banta, IUPUI |
2 |
|
5.
Daina Breidis,
Michigan State |
2 |
|
6.
Eleanor
Nault, Clemson |
2 |
|
7.
Dick Seagrave,
Iowa
State |
2 |
|
8.
People
at ABET (various people mentioned including Kate Aberle, Ellen Stokes,
Dan Hodge, Bob Herricks) |
2 |
|
9.
Ira
Jacobson, formerly at Embry-Riddle, former officer of EAC |
2 |
|
10.
Joseph Hoey, Georgia Tech |
2 |
|
11.
Neal Armstrong, UT Austin |
1 |
|
12.
Mary
Besterfield-Sacre, University of Pittsburgh |
1 |
|
13.
Patricia Daniels, Seattle University |
1 |
|
14.
Ron
DeLyser, University of Denver |
1 |
|
15.
Michael
Steven Leonard, Clemson – moved to
University of
New Mexico |
1 |
|
16.
Jack
Lohman (editor of ASEE Journal) |
1 |
|
17.
John
Lorenz, Kettering |
1 |
|
18.
Rob
Pangborn and Linda Strauss,
Penn
State |
1 |
|
19.
Sarah Pfatteicher,
University of
Wisconsin |
1 |
|
20.
Dr. Narayan Rao,
University of
Illinois |
1 |
|
21.
Joni Spurlin,
North Carolina
State |
1 |
|
22.
People
in larger field of assessment including Peter Ewell, Thomas Angelo,
Alexandra Astin |
1 |
|
23.
No
one/no one else comes to mind outside of college/school |
9 |
21.
Other
thoughts?
|
Comment |
# |
|
1.
Consider using a mock reviewer |
2 |
|
2.
Insufficient attention has been paid to ownership of lower division
courses |
1 |
|
3.
To the
extent that you can demonstrate that you are taking steps to improve
your faculty’s teaching skills, or improve their effectiveness in the
classroom, that is important to show the visitor. |
1 |
|
4.
Take
advantage of resources that are available on your particular campus. |
1 |
|
5.
I am
interested in Exam Café which is a practice tool for the FE exam. A
faculty member can create a piece of an FE exam focusing on a
particular topic area. You can use the results of this for
accreditation, and because it is sort of a mock-FE exam, it allows
students to see how they would do on the real FE exam. The cost for
students to take this mock exam is very low. Check out their website
at www.ppiexamcafe.com . |
1 |
|
6.
People
concerned about ABET fall into two categories – faculty (who either
accept and try to find the most efficient way to deal with ABET or
they hate ABET), and the Assessment community. In this last group,
there are many who are well-intentioned, but they see assessment as an
academic exercise in its own right. This latter group talks a lot
about what it takes to provide valid assessment tools, many of which
require the dedication of enormous amounts of resources. It boils
down to an optimization problem – how do you trade off assessment
investment against lab/curricular investments? |
1 |
|
7.
Many of
the evaluators at ABET are NOT from research intensive universities;
we sometimes suspect that they think we don’t care about education or
that they think “we’re going to take them down a notch and we’ll show
them how rigorous we can be about education”. There is a big concern
about whether or not you can get a fair assessment from someone who is
not from a similar institution. At research institutions, education
is not the only driver and our visitors need to understand that not
all of our resources can be directed to education. |
1 |
|
8.
Because
we were unable to predict where problems would arise, we decided that
it would be best to start off by figuring out what made sense for our
institution and then secondly, try to demonstrate the link to the ABET
criteria. This is important, because you can’t assure that you will
be able to convince ABET no matter what you do! |
1 |
|
9.
Current
ABET criteria have not made things easier – it forces you to be more
thoughtful. But it is much more painful now and I’m not sure that it
is that much better in terms of the results. |
1 |
Results by Question
1.
What advice
can you provide regarding how to conduct a successful ABET review (e.g.,
best practices you can share)?
|
Comment |
# |
|
1.
Be sure you can demonstrate that a continuous improvement process is in
place and that changes have been made as a result of assessment |
9 |
|
2.
Establish an ABET committee with representation from each department
for the purpose of coordinating and sharing best practices; meet
regularly |
8 |
|
3.
Collect minimal information to get maximum results – be sure it is
sustainable; effective and not burdensome |
5 |
|
4.
Develop clearly stated goals and objectives, mission statements, etc. |
4 |
|
5.
Have good documentation, which gives the evaluator a good feel for what
is going on in the program (mi | |