“We all want to belong, yeah …”

I just watched a wonderful TEDx talk by Terrell Strayhorn, Professor of Higher Education at (the) Ohio State University, called “Inalienable Rights: Life, Liberty, and the Pursuit of Belonging.” With enviable ease, Dr. Strayhorn walks his audience through the various factors that impede college persistence and demonstrates why a sense of belonging is so important for student success. He concludes his talk with his remarkably smooth singing voice, crooning, “We all want to belong, yeah . . .”

If you’ve been following my blog over the last year you’ve seen me return to our student data that reveals troubling differences in sense of belonging on campus across various racial and ethnic groups. The growing body of research on belongingness and social identity theory continues to demonstrate that the factors that shape a sense of belonging are extensive. While these complicated findings might gratify the social scientist in me, the optimistic activist part of me has continued to beg for more concrete solutions; things that individuals within a community can do right away to strengthen a sense of membership for anyone in the group who might not be so sure that they belong.

So here are a couple of ideas that poured some of the best kind of fuel onto my fire over the weekend: Micro-Kindness and Micro-Affirmations. Both terms refer to a wonderfully simple yet powerful idea. In essence, both concepts recognize that we live in an imperfect world rife with imperfect interactions and, if we want the community in which we exist to be better than it is (no matter how good or bad it is at present), then individual members of that community have to take action to change it. Applied to the ongoing discussion of microaggressions and their potential impact on individuals within a community (particularly those from traditionally marginalized groups), both ideas assert that there are things that we can do to emphasize to others that we welcome them into our community and reduce the existence of microaggressions. These actions can be as simple as opening a door for someone and smiling at them, making eye contact and saying hello, or engaging in brief but inclusive conversation. Instructors can have a powerful micro-affirmative impact by taking the time to tell a student who might be hesitant or struggling that you know that he or she can succeed in your class.

Researchers at the Higher Education Research Institute at UCLA have found that validating experiences, much like the micro-kindnesses and micro-affirmations described above, appear to have a significant impact in reducing perceptions of discrimination and bias. In fact, after accounting for the negative impact of discrimination and bias on a sense of belonging, interpersonal validations generated by far the largest positive effect on a sense of belonging.

Research on the biggest mistakes that people can make in trying to change behavior has found that trying to eliminate bad behaviors is much less effective than instituting new behaviors. Since individuals often perceive microaggressions to come in situations where a slight was not intended, eradicating everything that might be perceived as a slight or snub seems almost impossible. But if each of us were to make the effort to enact a micro-kindness or a micro-affirmation several times each day, we might set in motion a change in which we

  1. substantially improve upon the community norms within which microaggressions might occur, and
  2. significantly increase a sense a belonging among those most likely to feel like outsiders.

Make it a good day,

Mark

 

Rethinking our “competition” for future students

Welcome back! I hope you found a way to carve out at least a few moments of relaxation and rejuvenation during the holiday break. Of course, the phrase “holiday break” doesn’t mean nearly the same thing for everyone, especially this time of year. For example, the folks in admissions are in the midst of working their tails off. Nowadays, the mayhem of recruiting high school students to a private liberal arts college doesn’t take a holiday, ever.

Over the last few years, we’ve learned a lot about the nature of our “competition” for prospective students. Not so long ago, many of us might have assumed that a high school senior considering Augustana College would therefore have already limited their list of potential colleges to a set of small liberal arts colleges, mostly located in the Midwest. Several decades ago this assumption was almost always correct. However, these days we know that the majority of prospective students who consider Augustana tend to look hardest at Midwestern public or larger urban private institutions as they narrow toward their final choice, not other small liberal arts colleges. This knowledge has clearly helped us make a more convincing case for choosing Augustana College, since knowing which institutions we are competing against helps us make our case more precisely and concretely.

Over the last few years, we’ve heard rumblings about other looming competitors, mostly in the form of online colleges or MOOCs (massive open online courses). Fortunately, most of those up-and-comers have blown themselves up on their own launch pads. But the underlying assumptions that justify the continued quest to build similar launch pads might be the real “competition” that we need to understand most of all.

During the holiday break I stumbled upon an opinion piece that lays bare those assumptions in a way that is as explicit as it is cocky. Neil Patel, a bigwig in the online start-up and entrepreneur world (exemplifying his marketing chops with the hyperbolic clickbait headline “My Biggest Regret in Life: Going to College”) asserts that going to college was a waste of time and money because it didn’t teach him any of the things he needed to learn in order to succeed as an entrepreneur. He argues that his college classes were little more than instances of learning isolated facts, theories, and concepts solely to regurgitate them on a test or in a paper before the end of that academic term (sort of the academic equivalent of “lather, rinse, repeat”). He argues that the entire exercise fails an ROI (return on investment) analysis because he could have learned much more useful information, grown in more substantive ways, and ultimately made more money by diving into the real world right out of high school.

I am not sharing this article to suggest that Patel is right, although my own experience at big public universities as both a student and as an employee doesn’t do much to squash his argument. Rather, I share this article to lay bare the nature of our real competition. Because whether it is less expensive public institutions (2-year or 4-year schools), online institutions, some combination of MOOCs and competency-based education, or merely the simplification of a college choice to the largest financial aid package, in most cases our real competition isn’t other institutions. Instead, it is embedded in a series of assumptions that set up an entirely reasonable conclusion . . . IF those assumptions are, or appear to be, true. The logic stream goes something like this:

  1. College is primarily composed of a series of discrete experiences (AKA classes) that require regurgitating information that has been recently memorized.
  2. The information that is to be regurgitated exists in isolation (AKA is rarely transferable to other college experiences or to life after college).
  3. Accumulating completion approval (AKA at least a passing grade) for set number of classes across a set of categories earns a credential of completion (AKA a bachelor’s degree).
  4. Therefore, find the least expensive way to ensure a reasonable likelihood that one earns this credential.

The hardest part of facing the real world implications of this rationale is that we aren’t talking about our truth. We are talking about prospective students’ truth – the conclusions they draw as they take in what we tell them online, in print, and in person. This is the “truth” that drives real behavior. So as much as we might want to passionately argue that college transforms or that students just can’t know how what they learn will be useful until long after they’ve learned it, if the information that prospective students gather as they look at Augustana College doesn’t emphatically dispel the assumptions that undergird the logic stream spelled out above, all of our hot air (hot print, hot pixels, etc.) will likely end up sounding like a lone coyote howling at the moon.

The other hard part of facing this reality is realizing that prospective students apply this logic (fairly or not) in real time. So we help ourselves a whole lot when we show concrete evidence, from the very beginning of our interactions with each prospective student, that the experience we provide is not focused on memorizing and regurgitating information. And we help ourselves even more when we can show concrete evidence that the things students learn in one setting are directly applied during college and after college. Unfortunately, the lens through which prospective students increasingly evaluate potential colleges is not an unbiased lens. Rather, it is pre-tinted with the aforementioned assumptions, making it critical that every student sees in the most explicit and obvious ways that our understanding of a college education blows those pre-existing assumptions to bits.

All this leads to a pretty important question. If someone were to look at any of the documents or webpages that describe a given educational experience at Augustana (a syllabus, a program description, etc.), how would someone holding the assumptions described above respond? Is there a chance that the document or webpage in question would leave those assumptions unchallenged? Worse, would a review of those documents or webpages confirm those assumptions? Or would that document or webpage shatter those assumptions and open the door for a conversation about how an Augustana education might be completely different from anywhere else?

For those of us who aren’t on the front lines of recruiting students every day, this post might seem overblown. For the folks who are slogging it out in the trenches, this post might not seem urgent enough. But it seems pretty clear that these assumptions are driving the way that many prospective students and their parents start the college search process. If we don’t actively shatter those assumptions early and often, we leave ourselves susceptible to ending up on the short end of a flawed ROI argument. And to rub salt into the wound, if we end up on the losing end of this argument, we won’t even get the chance to challenge the flawed nature of their ROI analysis, because by then the prospective student has likely already crossed us off their list.

Sorry for the sobering post to start the new year. But sometimes sobering isn’t such a bad thing. In this case, we have the winning argument and the evidence to back it up. So knowing the nature of the “competition” gives us one more advantage that we ought to use every chance we get.

Make it a good day,

Mark

How do we improve a student’s sense of belonging?

For the last few years we’ve been talking a lot about our students’ “sense of belonging” after seeing some troubling differences between various student types. Although the overall scores might seem pretty good, stark differences between black and white students suggest a disturbing problem. Looking deeper, we found that hispanic male students also indicate a notably lower sense of belonging. We’ve since found indications that low-income students, first generation students, and lesser academically prepared students can exhibit signs of a lower sense of belonging as well.

Although this news has been tough to swallow, I’ve been really proud of the way that our whole community has committed to making Augustana a more inclusive place. This is a critical first step that shouldn’t go unnoticed, since there are lots of examples of places that have responded to this kind of sobering news by sticking their proverbial head in the sand (or snow, as the case may be). But finding answers to this challenge is complicated. None of our students fit into neat little exclusive categories like hispanic or low-income or first-generation or male. Instead, every student possesses some mix of characteristics that, taken together, uniquely affect the way that they experience Augustana. So improving any student’s sense of belonging means that we need to know a lot more about the perceptions that lie beneath this more general malaise.

Last spring several of my students and I decided to see if we could figure out a bit more about those underlying perceptions. Although there are probably lots of ways to tackle this challenge, after digging into the relevant research my student-workers and I decided to build a set of survey items derived from research on a concept called microaggressions. In short, microaggressions are expressions that communicate animus, aversion, or disregard toward someone specifically because of that person’s membership in a marginalized group. They can be verbal or nonverbal and are sometimes intentional and sometimes not. Although there are some legitimate critiques of the applications of the microaggression construct, this taxonomy of microaggressions provided a useful framework that aptly applied to our project. After testing and tweaking these items with a small group of students, we plugged them into the freshman survey that went out at the end of last year’s spring term. Each item was accompanied by five response options ranging from strongly disagree to strongly agree. The final list of survey items were:

  • I can learn anything if I set my mind to it.
  • I have to work harder to fit in at Augustana than most students.
  • People on this campus believe that I am just as capable as everyone else.
  • People on this campus believe that everyone has the same chance of making the most of their college career as long as they work hard.
  • I’ve gotten better at bouncing back after facing disappointment or failure.
  • Augustana students recognize discrimination when it happens on campus.
  • People on this campus seem to feel uneasy or nervous around me.
  • People on this campus do not seem to acknowledge the characteristics that make me different.
  • People at Augustana tend to assume that I come from a different culture.
  • More than once students on campus have made inappropriate comments or demeaning jokes about me or the group to which I belong.
  • Students at Augustana often make assumptions about me based on the way I look and dress.
  • More than once I have felt overlooked when trying to interact with faculty or staff.

Our first clue that we might be on to something came when we tested the correlations between each of these items and sense of belonging. In all but one case (“I can learn anything if I set my mind to it.”), the correlations were statistically significant and in several cases intriguingly large. (For all you stats nerds out there, by “intriguingly large” I mean approximately .3 and .4 or -.3 and -.4 depending on the phrasing of the item). Then, when we ran more elaborate regression equations that took into account race, gender, incoming ACT score, socioeconomic status, and first-generation status, we found that 10 of the 12 hypothesized sense-of-belonging predictors (all of the above items except “I can learn anything if I set my mind to it” and “People at Augustana tend to assume that I come from a different culture”) produced statistically significant results in the direction that we would expect. In other words, most of these items appear to capture some of the perceptions that underlie a reduced sense of belonging and, consequently, might also give us some hints about the ways that we could bolster sense of belonging among students who lack it.

Lastly, we noticed a curious pattern in our regression equations. In 7 of the 12 equations, race (coded as white/non-white) produced a statistically significant effect, and, in all 12 equations socioeconomic status (coded as receiving a Pell grant or not) produced a statistically significant effect. In other words, race and socioeconomic status consistently play a critical role in shaping a student’s sense of belonging even after accounting for each individual predictor above. So we conducted one more set of analyses to identify the items that might be most prominent in shaping sense of belonging for different types of students.

Although I’ll summarize what we found below, I’ve added a link to the full table of results testing differences by gender, race, socioeconomic status, first-generation status, and incoming ACT score (for clarity’s sake we compared the bottom third against the top third of incoming ACT scores). We’ve included the items where the difference between the two groups was statistically significant as well as the two instances where the difference was just a hair above the p=.05 threshold.

In essence, we found that differences on various items appear between groups across all of the pairings that we tested. In many cases, the differences played out as we would expect. Students of color exhibited disadvantaging self-perceptions on numerous items, particularly items addressing the assumptions (be they perceived or real) that others make about them. Students with lower incoming ACT scores also exhibited a number of disadvantaging self-perceptions. Moreover, “I have to work harder to fit in at Augustana than most students” and “People on this campus seem to feel uneasy or nervous around me” produced statistically significant differences across multiple pairings.

Interestingly, some results challenged prior applications of microaggression theory. For example, the differences between men and women clearly suggested that men potentially suffer from several disadvantaging perceptions. Contrary to the prevailing assertion that women would be the ones to exhibit lower self-perceptions, men scored lower on four items, most notably, “People on this campus believe that I am just as capable as everyone else” and “People on this campus believe that everyone has the same chance of making the most of their college career as long as they work hard.” And although students of color scored lower on a host of items than their white counterparts, they did score higher on the item, “I can learn almost anything if I set my mind to it.”

So what are we to do with all of this? Clearly, this analysis seems to suggest some useful hints about the types of students who might be susceptible to a lesser sense of belonging as well as some hints about ways that we could validate their membership in our community. For example, for students who might feel like they have to work harder to fit in, we can take the time to explain that with regard to academics, developing a robust workrate is a vital precursor to a successful life and if a students already finds themselves increasingly working hard, this may well mean that they are further along than many of their peers. Conversely, if their sense of working harder to fit in relates to their social integration, then we might just have carved out an opening to the kind of conversation or referral that could address this concern. I suspect that some reflection on each of the items noted in the full table might generate additional ideas about how to help students who find themselves wondering if they really belong.

One other implication of these findings seems worth noting. Much of the research on microaggressions has argued that evidence of differences in self-perceptions on items like these is likely, or even necessarily, evidence of discriminatory behaviors or beliefs on the part of members of the pairing who scored higher on that item. In some cases, maybe. But it seems that the pervasiveness of differences that we found across all of these pairings suggests that the factors contributing to a lack of belonging can’t be solely attributed to verbal and nonverbal, intentional and inadvertent slights, snubs, or insults. It’s likely much more complicated than that. It seems to me that this taxonomy of microaggressions is more useful in guiding the way that we might build up someone’s sagging sense of belonging than it is in forcing an interaction to be perpetually framed within the confines of a target/victim label. Intent is a dicey thing to presume, and although we certainly want to help our students understand the implications of their words, arguing about the intentions of another seems likely to become an unresolveable errand after which there is little chance of learning the greater lesson.

As educators, we are always striving for two simultaneous results:

  • to foster an ideal learning environment in the present, and
  • to prepare our students to succeed no matter what life throws at them in the future.

While we absolutely want every student to feel a similarly robust sense of belonging, and while we certainly want every student to feel a similarly minimal set of inferiorities and anxieties, I wonder whether we could ever achieve an ideal learning environment without moments of interactive difficulty that spawn feelings of self-doubt and uncertainty. In the Wabash National Study of Liberal Arts Education, we saw that institutions where students’ intercultural skills grew the most also reported higher frequencies of positive and negative diverse interactions. Certainly we will always need to teach students to think carefully about the import of their words, but I hope we can remember to balance our efforts to support our students in the midst of their hurt or offense with equal efforts to push, prod, and persuade our students to grow in the presence of difficulty. It would be in those moments that, when we support and challenge, we will most fully accomplish our educational mission.

Make it a good day . . . and a good holiday break,

Mark

Careful Planning of Course Offerings Seems to be Paying Off

Have you ever had one of those moments where you put a lot of time into something only to discover that you really just need to start over?  Well, that has been my experience this week in trying to write about some data that we collected from our freshmen last spring regarding the kind of perceptions that are often attributed to microaggressions. So instead of dumping a post on you that isn’t up to snuff (what does that phrase mean, anyway?), I’m going to take it back to the drawing board and post it next week.

In the mean time (cue the Jeopardy music), since many of you are working through course master planning for next year, here is a set of data points that ought to make you smile.

One of the more practical predictors of our seniors’ sense that they would choose Augustana again is the degree to which they found that the courses they needed to take were available in the order in which they needed to take them. Even though there might be a myriad of paths to complete one’s degree, it’s not too difficult to tell the difference between a student who can talk through how their classes fit together and a student who seems to have scrambled through their four years with little more than a grab bag of credits to show for it.

Over the last three years, our seniors’ average response to this item has gone from a 3.06 in 2014 to a 3.50 in 2016. I suspect that this improvement can be credited to improved course planning as well as improved advising. I don’t have a good sense of how that balance breaks down, but I think anyone who’s played a role in either aspect of helping students move through their four years in a more deliberate way deserves a small pat on the back.

So if you are slogging through course master planning ’til late at night and you wonder if it’s worth it . . . it might just be. And if you find yourself wondering if all the time you spend advising students is worth it . . . it might just be.

Make it a good day,

Mark

What do you do when change finds you?

Welcome back from the short holiday break. Don’t tell the wellness folks, but I hope you got to eat all the pumpkin pie and whipped cream you could stand!

But just in case you thought that you were going to ease your way back into the comforting routine of winter term, I thought that now would be the perfect time to tell you about a nifty change that is coming down the pike.

What if you could see your IDEA feedback summary and student comments as soon as you submitted your grades for the term? It seems to me like that would be pretty awesome.

And what if you could get real-time IDEA feedback from your students in the middle of the term so that you could adjust on the fly? It seems to me like that would be pretty cool, too.

Remember a year or two ago when I reported that IDEA was going to phase out their paper forms some time in the next several years? Well, I’ve been informed that the paper forms will breathe their last collective breath in the spring of 2018. That means that, unless we want to go out onto the market and audition all of the other players in the course feedback survey industry (from whom I get phone calls or emails at least twice a week about their “exciting fully customizable online format”) or take on the monumental task of building a homegrown course feedback system, we need to plan on moving to a paperless IDEA system in the fall of 2018.

To be honest, I may have oversold this change a bit. In reality, it’s not going to change the daily life of an instructor much. You’ll still choose your learning objectives at the beginning of the term, and the students still complete the same set of items (albeit with some improvements that actually align better with our own college outcomes) at the end of the term. In many cases, you’ll likely opt to use class time for students to enter their responses on their phones, tablets, or laptops instead of coloring in little circles on a piece of paper. Solving the potential problem that some students won’t have a device that can access the survey is probably a pretty simple one (one possibility would be to borrow a neighbor’s phone or laptop).

Now I suspect that some of you have questions about how this is going to work. And we wouldn’t be doing our jobs if we didn’t wonder about all of the ways that the paperless process could go horribly awry. But after quizzing the IDEA consultant, it seems as if they’ve found a myriad of ways to avoid the obstacles we most often worry about (e.g., low response rates, satisficing, etc.).

Nonetheless, we will host plenty of opportunities to answer questions about the details of this change – both in person and online. So what do we do when change finds us? Put your arms out wide and embrace the possibilities!

Seriously, what else are you going to do?

Make it a good day,

Mark

 

A Short Post for a Short Week!

Hello Everyone,

Since this week is a short one, and last week I rambled more than I should have, this week I’m going to keep it very short.

One way to assess the effectiveness of an office on campus is to look at how pervasively its services are recommended by others across campus.

We ask this very question of seniors regarding the ubiquity of recommendations they receive to use the services that CORE provides. If the frequency of recommendations is going up . . . that’s probably good. If the frequency of recommendations is going down . . . that’s probably not so good.

Here is the trend over the past four years of the proportion of seniors indicating that no one recommended CORE (or the CEC in 2013 and 2014) to them –

  • 2013  –  18.0%
  • 2014  –  18.9%
  • 2015  –  14.4%
  • 2016  –    7.1%

Looks like CORE is doing something right.

Make it a good day . . . and have a nice couple of days off.

Mark

The First Year Experience: A treasure trove and a quick peek behind the curtain

And they’re off!

It’s always a good thing when you can wear short sleeves, slacks, and sandals on the first day of winter term!  (I know nobody uses the word “slacks” any more, but I couldn’t resist the alliteration.)

Welcome back and good luck with the start of your winter term.  It was a pretty quiet break with nothing really going on . . . oh yeah, except for that.

Nonetheless, the IR office has finally finished putting together our two big reports from last year’s (2015-2016) Senior Survey data and First Year Survey(s) data.  These reports are both linked on the IR web page, so please help yourself to an overflowing spoonful of mean scores, standard deviations, frequency distributions, and bar graphs! (I know! So exciting.)  It’s a veritable smorgasbord of quantitative delectables.

In particular, if you scroll past the first nine pages of the 15-16 First Year Survey(s) Report, our student worker Katrina Friedrich create a table that highlights the statistically significant predictor variables of seven different intended outcomes of the first year:

  1. I feel a strong sense of belonging on campus.
  2. Over the past academic year, I have developed a better sense of who I am and where I want my life to go.
  3. If you could relive your college decisions, would you choose Augustana again?
  4. During the year I got better at balancing my academic with my out-of-class activities.
  5. I am certain that my choice of majors(s) is a good fit for who I am right now.
  6. How often did you push yourself to work harder on an assignment even though the extra effort wouldn’t necessarily improve your grade?
  7. I found myself thinking about what I am learning in my classes even when I’m not in class or studying.

Although I’m sure we will spend more time this year digging into the various findings highlighted in this table, this post wouldn’t be complete without at least one guided exploration into one of the predictor variables that just keeps popping up.  So today I thought we’d kick it up a notch “Bam!” by exploring the backstory of one pesky predictor variable for first year students:

  • “Reflecting on the past year, I can think of specific experiences or conversations that helped me clarify my life/career goals.” (response options ranged from strongly disagree to strongly agree)

This item turned out to be the only variable that significantly predicted all seven of the outcome variables. (For all of you who gobbled up the nerd salad a long time ago, each of our regression equations included controls for race, gender, socioeconomic status, and pre-college academic preparation). So the next question seems pretty important: What specific experience(s) might statistically predict students’ ability to recall specific experience of conversations that helped them clarify life/career goals?

Just like the analyses that Katrina conducted to produce the initial table of results, to run a reasonably legitimate test I build a regression equation that took into account race, gender, socioeconomic status, and pre-college academic preparation. The basic reason to include these variables at the outset is to allow us to say with confidence that our findings apply to all students regardless of differences that might exist across those four demographic characteristics.

Then I added nine variables that might be relevant considering what other researchers have found about what might help a student find some clarity of purpose. The items I chose to add to this analysis are:

  • My first year adviser asked about my career goals and post-graduate aspirations.
  • My first year adviser connected me with other campus resources to help me thrive in college
  • My first year adviser recommended specific on-campus activities to help me make the most of my college career.
  • My first year adviser pushed me to think about choosing courses as more that just checking boxes.
  • My out-of-class experiences involved me in community service off-campus.
  • My out-of-class experiences helped me connect what I learned in the classroom with real-life events.
  • How often did your instructors ask you to apply your learning to address societal problems or issues?
  • My on-on-one interactions with faculty have had a positive influence on my intellectual growth and interest in ideas.
  • Symposium Day activities influenced the way that I now think about real world issues.

Would you like to venture a guess which items popped (not the technical term, I know, but I’m trying to encourage some new slang amongst my people)?

Listed from largest to smallest effect size, these three items produced statistically significant positive effects.

  • One-one-one interactions with faculty positively influenced intellectual growth and interest in ideas.
  • Out-of-class experiences helped connect classroom learning with real-life events.
  • First year adviser recommended specific on-campus activities that would help make the most of one’s college career.

Refreshingly, these findings suggest that all of us can play a potentially key role in helping our first year students clarify their life and career goals.  If you interact with students in a faculty role, then look for ways to create one-on-one interactions that engage substantive questions. If you interact with students outside the classroom, look  for ways to help them connect their academic learning with real world events.  And if you interact with students as an adviser, then make the effort to identify and recommend specific on-campus activities that align with, and might even augment, your student’s post-graduate aspirations and college goals.

Although we can’t guarantee that a student makes the most of their college experience, we can increase the odds that he or she chooses the behaviors and activities that will point them in the right direction.  And if we keep it up long enough, we will likely be a pretty damn good choice for the students who are lucky enough to come here.

Make it a good day,

Mark

It’s Hard to Argue with this Welcome Week Data

Good Morning!

It’s week 10!  The last week of the fall term!  You can make it!

This week I’d like to send a virtual shout-out to all of the folks who run Welcome Week for our new freshmen at the beginning of the fall term. This four-day whirlwind is a logistical Cirque du Soleil of social and academic acclimation.

But in many ways, it’s really more of an orientational triage. There are certain things that the students have to know by the time classes start or they’ll tank right out of the gate. Then there are other things that we’d love them to learn but we know these things might be a bridge too far. In reality, four days isn’t a lot of time, and the students’ ability to digest information is undercut by all of the anxieties that come with knowing, “Holy crap, I start college in a few days!” So the Welcome Week design team is faced with a stark reality: be very clear about the difference between what these new students have to know and what would be nice to know. Then teach them all of the first category and as much of the second as possible – knowing that too much time spent on any of the “would be nice to know” could cut into the “have to know” and then we’ve got a potential problem.

A few years ago, I highlighted the ways that the Welcome Week team has used some simple assessment design principles to improve the quality of the experience. But in that post, we only had anecdotal data to suggest that some good things were happening as a result. Now that we have a couple years of quantitative data, the evidence is pretty clear: Welcome Week has gotten even better at doing exactly what it is supposed to do.

A few weeks after the beginning of the fall term, we ask freshmen to complete a short online survey to find out their perception of Welcome Week. Specifically, we want to know the degree to which they think they learned the things we tried to teach them. I’d like to highlight four items that represent things that we think students have to know. Below each item is the average response score on a 1-5 scale (1=strongly disagree and 5=strongly agree) from each of the last four years. Notice the steady improvement.

My Welcome Week experience . . .

. . . helped me learn exactly how to get to the location of my classes.

  • 2013 – 3.55
  • 2014 – 3.79
  • 2015 – 4.18
  • 2016 – 4.21

. . . helped me find places on campus where I can study most effectively.

  • 2013 – 3.59
  • 2014 – 3.63
  • 2015 – 3.82
  • 2016 – 4.00

. . . taught me specific ways to make the best use of my time during the school day.

  • 2013 – 3.23
  • 2014 – 3.39
  • 2015 – 3.40
  • 2016 – 3.68

. . . emphasized the importance of finding places on campus where I can take time for myself.

  • 2013 – 3.51
  • 2014 – 3.60
  • 2015 – 3.69
  • 2016 – 3.84

As you can see, the Welcome Week team deserves some well-earned praise. They’ve stuck to the overarching design and philosophy of the program and used evidence to inform change. They have redesigned several parts of the experience, revised the way that they train peer mentors, and tackled some difficult logistical challenges to ensure that our new students are more likely to be as ready as possible for the first day of classes. Equally difficult (and probably even more impressive), they’ve stopped doing a number of things – no matter how strongly they believed in the potential of those activities, in order to concentrate more precisely on making the most of every minute of those four days.

Late last week I was playing with our freshly-collected freshman data from the end of the first term to see if we could see any lasting effects of the Welcome Week experience. As you might expect, the impact of Welcome Week tends to fade as subsequent fall term experiences become more influential in driving student success. However, one particularly gratifying finding popped when I tested whether any of the Welcome Week survey items might predict our students’ response to an item in the end of the first term survey, “Welcome Week provided the start I needed to succeed academically at Augustana.” Even though the data collected from the Welcome Week survey was gathered during the second week of the term and the end of the first term data was collected during weeks seven and eight, the item “My Welcome Week experience taught me specific ways to make the best use of my time during the school day,” proved to be a statistically significant positive predictor of our freshmen’s perception of the preparatory effectiveness of Welcome Week. Impressively, this is also one of the learning goals where the Welcome Week team seems to have made substantial strides in preparing our new students to succeed.

So congratulations to everyone involved in putting together and pulling off Welcome Week!  I hope you’ll take a moment to send a kudos to anyone you know, even yourself, who contributed to a great Welcome Week way back at the beginning of the term.

Make it a good day,

Mark

Men, Social Responsibility, Volunteering, and Some Troubling Data

Last week I shared the first round of findings from our study of the 2012 cohort’s intercultural competence development during their college career. One finding that jumped out was the disappointing difference in change between men and women. While women’s scores improved on both the cognitive and the behavioral scales, the men’s scores only improved on the cognitive scale. In addition, the women’s improvement on the cognitive scale was notably larger than the men and the degree of women’s improvement on the behavioral scale almost doubled the advantage they started with over men four years earlier.

At the Board of Trustees meetings last week, I provided our annual Academic Quality Markers for the 2016 cohort to the Academic Affairs Committee. It’s pretty apparent that there is something troubling going on with male participation and engagement. Male participation in study abroad, service learning, and volunteering is significantly lower than women. This pattern continues in three student experience items that address our efforts to cultivate citizenship. Moreover, the other comparisons by race/ethnicity and socio-economic status don’t contain such repeated disparities between groups. The only other significant difference occurs where one would expect: white students report less encouragement to interact across difference compared to students of color. Given the substantially higher proportion of white students on campus, it would certainly take relatively less “encouragement” for students of color to find themselves interacting across difference.

I’m sure that the explanation for these differences between men and women are complex. However, we might have found something that could enlighten an effort to better educate our male students within the Global Perspectives Inventory (GPI) data that I shared last week and referenced above. One set of questions within this survey, the Social Responsibility Scale, is composed of statements that focus on the degree to which the respondent engages in the public sphere to affect change. As an example, two of the statements (to which the respondent indicates a level of agreement or disagreement) are: “I work for the rights of others,” and “I consciously behave in terms of making a difference.”

It might not surprise you to find out that male and female Augustana students from the 2012 cohort entered with different average scores, different enough that the gap would be considered marginally statistically significant.

  • Female: 3.76
  • Male: 3.62

But what surprised me was that over the course of four years, only the women had grown on this scale. Male students had on average remained unmoved.

2012 females: 3.76 – – – – – – – – – – – – 2016 females: 3.88

2012 males:    3.62 – – – – – – – – – – – – 2016 males:    3.59

Maybe this lack of male growth in prioritizing social responsibility partially explains the difference between men and women in volunteering and service learning participation. Maybe it partially explains the male deficit in getting something substantive out of Symposium Day. And maybe it partially explains the relatively lower sense among men that Augustana encouraged them to interact across difference.

If our goal, as our mission statement seems to suggest, is to graduate individuals who engage in both leadership and service, it appears that we may need to revisit the ways that we develop a service orientation among our male students.

Hmm . . . if only there were a major reconfiguration of the Augustana educational experience that would allow us to try something new based on these findings . . .

Make it a good day,

Mark

Does Augustana students’ intercultural competence improve during college?

In the fall of 2011 we set in motion a college-wide assessment plan where we would collect learning outcome data from each entering cohort, link this data to the various student experience surveys these students complete at different points during their four years at Augustana, then collect the same learning outcome data just before the cohort graduates. This plan allows us to track our students’ four-year change on a specific learning outcome and identify connections between student experiences and variations in the direction and degree of that change.

Obviously it would be logistically impossible (and a little stupid) to tackle all of the Augustana Learning Outcomes every year. So we decided to rotate annually through the three broad categories of learning outcomes starting with intrapersonal conviction, moving to interpersonal maturity, and finally addressing intellectual sophistication before returning to intrapersonal conviction. Because each category includes a variety of more specific outcomes, this framework allows us some flexibility in selecting outcomes that seem particularly pertinent to our students’ success while maintaining a more general pattern that keeps us tuned in to the totality of our learning goals.

The first cohort (starting in the fall of 2011 and graduating in the spring of 2015) provided data on orientations toward different types of motivation, something that undergirds the learning outcome that we have called “Wonder.” I wrote about some of our findings from that study last fall and last winter.

The freshmen who started in the fall of 2012 completed a survey called the Global Perspectives Inventory, an instrument designed to measure intercultural competence (an important aspect of the learning outcome category we call Interpersonal Maturity). In the spring of 2016 we collected the final set of data from this cohort. On September 16th, the Assessment for Improvement Committee (AIC) presented the first of three Friday Conversations (one in each term during the 16-17 academic year) intended to examine this data and explore what it might suggest. For those of you who were unable to attend the Friday Conversation on September 16th, I thought I would post the power point slides below. They give a brief description of intercultural competence, convey the nature of our students’ change on three aspects of intercultural competence as measured by the GPI, and pose some questions for us to begin thinking about what we might explore in preparation for our winter term Friday Conversation.

So click on this presentation of 4-year change at Friday Conversation 9/16/2016 and you will be able to scroll through the power point slides.

As you can see, we found that our students (at least this cohort of students) grew on two of the three elements of intercultural competence. Our students grew the most on the cognitive scale that assesses knowledge of cultures and the implications of differences between cultures. Our students also grew, albeit to a lesser degree, on the behavioral scale that attempts to capture the likelihood to enact behaviors that reflect intercultural competence. Finally, we found that our students made no statistically significant gains on the affective scale that assesses the attitudes that would motivate one to be intercultural competent.

In addition to examining the overall change, we also explored the change among several subgroups of students based on pre-college demographic characteristics. As represented by the bar graphs on several slides, this exploration discovered interesting differences in the intercultural competence growth between men and women, white students and students or color, and students whose ACT score suggested low and high academic preparation.

Reflecting on the changes that we see in our student data, the important next question becomes, Why? Why do our students grow in the way that they do?  Why do some students change differently than others? What experiences influence positive or negative changes in intercultural competence? In my mind, these are the more interesting questions to explore because they can point us toward concrete ways that we might improve the education we provide.

Of course, there are an almost infinite number of questions that we could ask of our data. Are there specific experiences from participating in distinct activities that improve intercultural competence? What about the possibility that a combination of experiences (especially in a specific sequence) might do more than any single experience? Finally, is it possible that a particular dynamic that pervades one’s college experience might transcend an individual experience or combination thereof?

Although we were able to solicit a long list of research questions to test from the folks in attendance at our first Friday Conversation, I’m sure there are many more that we have yet to consider. So please add a research question or two in the comments section below.  We will test as many as we possibly can.  And we will report back at the winter Friday Conversation and on this blog all of what we find.

So put on those hypothesizing caps, and send us your suggestions. If we can find a way to test it, we will!

Make it a good day,

Mark