There are four approaches to the assurance of student learning:

            1) Selection (have a very selective admissions policy)

            2) Indirect measures of learning (surveys)

            3) Demonstration (tests, performances)

            4) Course-embedded measurement by instructors

 ---------------------------------------------------------------------------------------------------------------

The College of Business cannot use the first approach but has often used indirect measures and demonstrations to show student learning, and has been learning to use the fourth approach of course-embedded measurements


Survey measurements
The College of Business realized the importance of having an ongoing assessment effort as early as 1994 when it created its Outcomes Assessment Committee for the College.  In its early years the Committee conducted focus groups and did surveys of undergraduate seniors, graduate students, employers and alumni to measure the effectiveness of the business curriculum and other business programs.  Focus group feedback and survey results pointed to the need for a center of advising that would be open every weekday for drop in help for students.  These findings lead to the creation of the Business Student Services Center in 1999.   Follow-up studies indicate that the Center has been a success for students and faculty.

 
The Committee continued a senior opinion survey that had been administered since 1992 to graduating seniors and graduate students.  This questionnaire asked students questions about their satisfaction with various aspects of their academic program and their perceived value of their experience in the College.  The results were reported to the Dean and the Department Chairs each fall.  This survey was discontinued in 1998 when direct testing of seniors began with the E.T.S. Major Field Test in Business.

 
A survey of College alumni who graduated in the 1992—1996 period was conducted by mail in fall 1996.  Over 600 alumni responded.  The results were reported to the College in spring 1997.  Both students and alumni were critical of the career center and related services.  In response the College created a Business Relations Center to serve just business students in contrast to the SFSU Career Center that tried to serve all majors on campus plus alumni.

There was also an employer survey conducted in 1996, a faculty survey in 1997, and a pilot study in 1997 of a case analysis given to BUS 690 Seminar in Business Policy students and cross-graded by two faculty members.  These last three efforts were disappointing.  The employer survey had sampling plan problems and a small sample size of just 22.  The faculty survey provided little useful feedback, and the case analysis approach was too labor intensive and also provided little useful feedback.

SFSU Exit Survey
In 1999, the University began administering a questionnaire to students filing for graduation.  In 2003, the Assessment Committee learned of these data and obtained the responses by business majors.  These responses provide data over four years as to the attitudes and opinions of our students.  For example, business graduate student were asked to rate the Faculty awareness of new developments in your field.  The proportion responding with Excellent (top rating in a four point scale) increased from ten percent in 2000 to twenty-one percent in 2003.

Undergraduates respond to SFSU Exit Survey

Undergraduate business majors respond to the item asking for their satisfaction with the Overall quality of instruction.  Sample size ranged from 211 to 513.

1= very dissatisfied 2=dissatisfied 3=satisfied 4= very satisfied (graphs show 3 + 4)

Required Courses Offered with Reasonable Frequency
Sample sizes ranged from 198 to 516.

My experience in my major program developed or enhanced my written communication skills.  Sample size ranged from 202 to 510.

1=not at all     2      3      4 = a great deal  (graph shows 3+4)

These results and others were reported to the Dean and the Department Chairs in spring 2004 by the Assessment Committee.  The upward trend in the percentage of undergraduates satisfied with the required course offerings is likely due to a new College policy of offering more sections on different days and at different times that was implemented in fall 2002.  On the other hand, the lower ratings regarding the enhancement of written communication skills may be due to the increase in class sizes that occurred during the budget cutbacks after 2000.  The Assessment Committee intends to study these data in conjunction with data on writing achievements by business majors to form a complete picture before recommending changes in curriculum or pedagogy.

E.B.I. Survey
In the spring of 2002, 2003, 2004 and 2006 a survey of seniors and graduate students was conducted.  The questionnaire was written and analyzed by E.B.I., a private educational measurement firm that does these types of surveys for colleges all over the United States.  Sample sizes have been about 200 undergraduates and 75 graduate students.  The questions focused on satisfaction levels of various aspects of the program including advising and perceived value of the experience.  Each September the Dean and faculty are briefed on the results.  Some results are posted at the Assessment Committee’s web site: http://online.sfsu.edu/~perttula/assessment/

The items on which students rated the College very high and very low provide good clues as to where improvement is necessary.  The top 15 rated items and the bottom 15 rated items are posted..   After significant changes in the College, we look to see if satisfaction scores rise.  We also examine the items to see if the top and bottom lists contain the same items year to year.

ETS Major Field Test

The Education Testing Service (E.T.S.) Major Field Test in Business was used during the 1998---2003 period to try to measure students’ learning and provide useful feedback to the faculty.  Students tested in IBUS 330 International Business represented entering business majors and exiting senior students were tested with the same test in BUS 690 Seminar in Business Policy.  There were significant problems with the length of the test (2.5 hours) and the motivation of the students to do their best.  When the administration went reasonably well, the data showed that there was a substantial increase in test scores for the exiting students compared to the entry students.  This increase could not be broken down by the concentration of the students so the results were of very limited use in improving curriculum.  

Business Assessment Test   

Dissatisfaction with the E.T.S. Field Test’s length and the motivation of the students to do well lead to several California State University Business Schools getting together to create their own test of basic business subjects.  The test was pilot tested in fall 2003, revised, and administered at several schools in spring 2004.  The College administered the test to 111 senior business majors in three sections of BUS 682 Social and Political Environment of Business in April 2004.  The results should provide some useful feedback as to the level of student learning in the various functional areas of business as reflected in the College’s required undergraduate courses.  The results of the analysis will be shared each September with the Department Chairs, the Dean, and core course curriculum coordinators.  The results will be analyzed by student concentration to see if there are differential weaknesses in subject knowledge by concentration.  Departments can then make changes in their curriculum as needed.  These tests will be administered annually to track the effectiveness of the changes.

The test was given in spring 2004 to 111 students who were in two day sections and one evening section of BUS 682  Seminar in the Environment of Business.
The test was given in spring 2005 to153 students who were in sections of BUS 690 Seminar in Business Policy and Strategic Management, our capstone course.
The test was given in spring 2006 to123 students in sections of  BUS 682  Seminar in the Environment of Business.  
The test was given in spring 2007 to 174 students in sections of BUS 690 Seminar in Business Policy and Strategic Management, our capstone course.
The test was given in spring 2008 to 95 students in sections of BUS 690 Seminar in Business Policy and Strategic Management, our capstone course.
The test was given in fall 2008 to 223 students in sections of BUS 690 Seminar in Business Policy and Strategic Management, our capstone course.

Department level test
In the late nineties, some interesting assessment work was being tried at the Department level.  The International Business Department, for example, created its own test of knowledge and skills and administered it to seniors in its IBUS 690 Global Strategic Management course.  Results of the test were shared among IBUS faculty each year.  The data pointed to the idea that students were forgetting many of the basic ideas and facts presented in the introductory course.  The faculty resolved to repeat these ideas and facts in all the international business courses so that students would be more likely to know them at the end of their course of studies.

Course Embedded Assessment
The AACSB recommends the assessment of student learning by a method called “course embedded” measurement.  Some faculty in the decision sciences and marketing departments learned this technique and practiced it in fall 2003 and spring 2004.  The technique is very efficient because it is very similar to the grading process that most instructors already know and use and does not require additional graders.  It can be effective at measuring student learning in a way that provides useful feedback for other faculty teaching in the required courses. The concept of a rubric must be learned and applied to the assignment that is embedded in the regular coursework. The College intends to expand the use of this assurance of learning technique to many more faculty.

Use of an oral presentation rubric in the graduate course, BUS 890, began in spring 2007 has continued each term since then including summer 2008.

Use of an oral presentation rubric and a writing rubric began in BUS 360 in summer 2008.

Assessment techniques and instruments in place in 2003-2004:

■ E.B.I. surveys (undergraduate students and alumni; graduate students and alumni; faculty)

■ Business Assessment Test (given to undergraduates in spring semester)

■ Course embedded measurements (being tried in two departments; expanded to five departments in 2004-2005)

■ Knowledge Test given by the International Business Department to its seniors

■ SFSU Exit Survey done every spring (approximately 400 business majors respond)