A close friend of mine believes successful PhD students have three things in common: They're smart, they work hard, and they have good judgment. The secret sauce here is "good judgment." Although smarts and hard work are important, most PhD students would have never been admitted if they weren't already smart, hard workers.
But having good judgment is more elusive. It includes things like knowing what's "interesting" and what isn't, knowing what's worth worrying about (and what isn't), knowing what's important to prioritize, knowing how to solve a people problem, knowing whether to persist on a project or to move on, and so on.
But advising a PhD student to "Have great judgment" is like advising a football team to "Score the most points." It doesn't tell them how. You can't say "Have great judgment" and then say "Next question," "QED," or drop the mic and walk out.
Maybe there's two types of judgment -- technical judgments and nontechnical judgments.
For graduate students, building technical judgment is about learning the whys of research. One way to build better technical judgment is to boldly ask lots of "Why?" questions of your mentor, advisor, or of an older student: "Why did you send it that journal? Why didn't you use a different method? Why did you ask the research question that way?" Most of us shied away from asking technical judgment questions because we didn't want to be irritating or look like we didn't belong. Most professors I know actually like to answer these questions, and they love to see an engaged student step out of a silent huddle.
Developing good nontechnical judgment is trickier. Yet this is the critical judgment you need to troubleshoot how you can be a better teacher, or whether to choose the risky dissertation you want to do versus the safe dissertation your advisor wants. It involves figuring out how to deal with your off-the-chart stress level or whether you should take a job at a teaching college or go into industry. Our tendency as a graduate student is to get feedback from peers in our same year. A more effective one may be to get it from recent graduates or from professors who have seen cases like these and know how they worked out. You can even get nontechnical advice from professors you know in other departments. The best nontechnical dissertation advice I got was from a Medical School professor from my church. It was straightforward, unbiased, kind, and based on lots of students he had known.
As professors, we can help to build better technical judgment by encouraging questions about our research judgment calls, or we can give it as color commentary or as context when we discuss a research project. But again, helping students with nontechnical judgments is trickier. One way to do this is in the third person. This can be by discussing a problem that "their friend" is having or by discussing a relatable case study.
Here's one approach to building nontechnical judgment. I used to teach a PhD course where we'd meet in my home for a casual last class session. The first half of the session would be a discussion about graduate student success and the last half would be dinner. Each student had been asked to anonymously write down a dilemma that "one of their friends" was facing that was being a roadblock to their success. We'd mix these 9-10 dilemmas up, and we'd relax in the living room with a glass of wine and discuss them one at a time. For each one, we'd talk about similar experiences, options, solutions, and so forth. By dinner time, we had a more balanced perspective and some suggested next steps for many of the dilemmas.
Over the years, it seemed that about 70% of these dilemmas were about the same 7-8 issues. These were like the issues mentioned above -- "risky vs. safe dissertation," "stress level," "leave academia," and so on.
Here's a second approach to building nontechnical judgment. Given how similar these dilemmas were from year to year, I wrote up 1-page PhD student case studies that involved slightly fictionalized people who were facing these common problems. These case studies were in the syllabus for the course, and we'd take the first or last part of each class to talk about that week's case study. The common dilemmas faced in your field may be different, but the enthusiasm your students would have in discussing them would probably be the same.
Some people might be born with great judgment, but for the rest of us, it's a lot of trial and error and a lot of asking bold questions. If you're a graduate student, you've got a lot more license than you might think to learn from trial, error, and bold questions. If you're a professor, there's a lot we can do to help them.
On a late afternoon about 20 years ago, I stepped into a slow elevator with my college’s most prolific, famous, and taciturn senior professor. After 10 seconds of silence, I asked, “Did you publish anything yet today?” He stared at me for about 4 seconds and said, “The day’s not over.” Cool . . . very Clint Eastwood-like.
As academics, we have great, productive days and we have bad days, but most lie in-between. If we could figure out what – other than an “Accept Without Revision” letter – leads to great days, we might be able to trigger more of them in our life.
Think of the most recent “great day” you had. What made it great, and how did it start?
For about 20 years, every time a colleague told me they had a great day, I’d ask “What made it great? How did it start out? About 50% of the time its greatness had to do with an external “good news” event like a paper getting accepted, a grant getting submitted, a great class, or a gracious letter from a former student. The other 50% of the time, the reason for “greatness” was more “internal.” They had a super productive writing day, they finished a paper, or they had a breakthrough idea.
External successes are easy to celebrate with these colleagues. Internal successes are more ephemeral, so it’s interesting to explore what was the trigger that made today a great day and what was it that sabotaged yesterday.
When people had great days, one reoccurring feature was that they started off great. There was no delay between when they got out of bed and when they Unleashed the Greatness. People said things like, “I had this idea last night, and I just got up and started writing,” or “I started on this revision right when I got up and had it half done by breakfast time.”
One of the most productive Grand Ole Dads in my field told me that he got up six days a week at 6:30 and wrote from 7:00 to 9:00 without interruption. Then he kissed his wife good-bye and drove into school. When I asked how long he had done that he said, “Forever.”
About a year ago, I started toying with this hypothesis: "Your first two hours set the tone for the whole day."
Think of your last mediocre day. Did it start out mediocre? That would also be consistent with this hypothesis.
We can’t trigger every day to be great, but maybe we have more control than we think. If we focus on making our first two hours great, it might set the tone for the rest of the day.
What we need to decide is what we can we do in those first two hours after waking that would trigger an amazing day and what would sabotage it and make it mediocre. For me, it seems paper writing, editing, exercise, or meditation are the good triggers, and it seems answering emails, reading the news, or surfing are the saboteurs.
Here’s to you having lots of amazing days. One’s where you can channel your best Clint Eastwood impression and say, “The day’s not over.”
It's been said that the most frequent last words of adventurous, partying males are probably:
1) “Hey, watch this,” or
2) “Here, hold my beer.”
If we heard either of these, our grizzled wise advice would probably be, “STOP, Don’t Do That!” But giving well-intended advice in less obvious situations is trickier, so we've grown more hesitant to do so. We’ve all been burned by giving advice and having someone either trigger their Eye of Death, or break a saloon chair and an unlabeled whiskey bottle over our head.
As a result, even as professors we can start drifting toward taking a more laissez-faire role toward advising students about their future. We might say “I will give them advice if they ask.” Yet even if they ask "What do I do?" we can be too carefully non-committal in giving them any advice (“Well, what do YOU want to do?”).
Last week, I had an interesting conversation with a person who said his son had been adrift in high school. It all turned in the right direction for him one day when a teacher who he casually played chess with said, “If you work hard, you could be a high-school chess champion.” He focused, and it happened. The Dad then said something similar had happened to him 50 years ago. He had been adrift in high school – good grades but adrift – when a teacher told him “If you work hard, you could be on the debate team.” He focused, and it happened.
These two teachers had given each of them a specific vision of what they could be: A chess champion and a debate champion. These mentors didn’t just compliment their talents by saying “You’re sharp,” or “You talks good.” They gave a specific direction that an adrift student could paddle toward. The decided to Be the One who pointed them toward an island.
With earnest students, it can be easy to say “Good job,” or “You’re creative,” or “You’re good at this class.” Those are compliments. Other types of compliments can give useful paddling directions. A student might be earnestly good at school but not see where to take their life other than in the general direction their parents, friends, or placement office talk about.
Suppose we took the risk that those two teachers took, and we told a student “You’d make a great ________,” or “Have you ever considered ____; I think you’d be really good at it.” They might feel a bit flattered, and a bit motivated to paddle in a direction they hadn’t thought of. Even if go in a totally different direction, if we motivated an earnest student in any hopeful direction, we accomplished more than if we would have given an easier default answer like, "Well, what do YOU want to do?"
Let’s circle back to last week’s conversation about the two teachers who stuck their necks out and make laid out specific visions to the guy and his son. Things worked out for both of them. Ten years later, the son had graduated from college, started his own business, and was coaching chess champion hopefuls on the side. Forty years later, the dad had retired as a Fortune 500 CEO to produce a movie. Partly because two mentors decided to Be the One who gave them direction.
I’m guessing that neither of their two teachers is still picking saloon chair splinters out of their head.
There's a reemerging teaching movement around this Be the One notion. Although it's sort of aimed at teachers of younger students, a surprising amount of it still applies to college students, and also applies to taking an extra effort to say the right words to graduate students at the right time. If you're teaching or TAing this semester, you can check out Ryan Sheehy's Twitter for a booster shot.
This season starts the beginning of job market Speed Dating in lots of academic fields. It’s when all of the schools who have jobs and all the PhD students who want jobs get together at their annual job market conference and speed date.
If a speed date goes well, a school will call you in about two weeks and ask you out on a campus date. If that 2-day campus date goes well, you get the engagement ring and get the job. If the 7-year engagement goes well, you get married with tenure.
But this all starts with the 45-minute speed date at the conference where there’s about a 1 in 3 chance that you’ll get called back. Two common questions: 1) How do I ace a conference interview, and 2) how do I know if I aced it?
Your advisor and friends have given you a big list of “dos and don’ts” for your interviews. Things like do act interested and do know lots about their school, and things like don’t act smarmy or arrogant and don’t dress like Spiderman or your favorite D&D character.
There's no perfect way to predict whether you’ll get a campus visit, but you hear lots of rules of thumb:
• The best day to interview is the second morning.
The next best is the first afternoon.
• The two best interview times are either 10:00 AM or 1:00 PM
• The more interviewers in your room (vs. skipping),the better your chances
• The more “fun” the interview seems, the better your chances
• The more questions they ask about your dissertation, the better
When I was on the rookie job market, I thought it would be useful to know which of these was the best predictor of a call-back. If a person knew that, they’d know when to schedule interviews with their favorite schools, whether to be serious or funny, and whether to encourage lots of dissertation questions. A few of my friends got together, and we came up with a list of about ten things we thought would predict how well an interview went. We all then rated each one of our interviews on these 10 things.
We then pooled everything together and ran a logit regression on whether we ended up getting a campus interview date at each school. Only 1 thing was significant. It wasn’t the timing of the interview or our subjective rating of how “fun” it was. Instead the only predictor of whether we got a campus visit was how many questions they asked about our dissertation during the conference interview.
Makes sense – except this was negatively related. The schools that asked us lots of questions about our dissertations DIDN’T fly us out for a date.
At the time, we thought maybe they realized we were doing a lot of handwaving in our theory section. Or maybe the deeper they dug, the more holes they found that we hadn’t yet plugged.
I now think there is another explanation. If we spent 35 minutes of a 45-minute interview talking about only 1 thing, the school only has 1 thing to judge us on (other than our advisor’s letter). It’s like a speed date where the person spends 90% of it talking about their vacation to Hawaii or their doily collection. We end up learning a lot about Hawaii or doilies, but not enough about them to want to call them back for a date.
A few years later, I was unexpectedly on the job market again as an assistant professor, and I did things a bit differently. Instead of spending 35 minutes droning on about a single project or two, I wanted to make sure we talked about a lot more than just Hawaii. That way, a broader set of connections could be made, and the people who hated Hawaii might find other things they could be interested in.
For all of the adrenaline-pumped PhD students at their job-market conference, it’s probably not the best strategy to spend the whole time talking about only your dissertation or only about Hawaii. The more other “good fit” connections you can make in the interview, the more likely you might be called back for a first date.
At this point, no last-minute finishing touches on your dissertation rap will help you ace your interview tomorrow morning. Instead, tonight's better spent learning more about your potential date – especially any unique overlaps you might have in common.
Good luck on the job market!
It was time to move this website over to this new account. Good news
It means having to rebuild every page from scratch. Bad news.
The last three or four months of blogs are being transferred over and posted. Ones that are older than are available here (for now).
Thanks for moving with me.
Most grad school application deadlines are due around January 1 and after that they won’t reopen again for another year. If you’ve ever walked the halls singing “Yo ho, yo ho, a PhD life for me” in a pirate accent, it’s not too late to make the big decision: Should you ruin your Christmas vacation by working feverishly to rush out a less-than-perfect application, or should you wait until next year?
If you really need the break next year and you’ve already got an incredibly awesome gig lined up, cool. Otherwise, take a deep breath, and take the 3-point shot. First, there’s no such thing as a perfect application – even if you wait a year. Second, you’ll either get accepted (a big pirate “Yo Ho!”), or you’ll learn what you need to do to improve your shooting percentage.
Here’s some super-short answers to some questions you might have:
• “How can I afford it?” Most PhD programs have assistantships that pay your tuition and living expenses.
• “Should I retake my GRE test?” Probably not enough time. If you think you have a good score, but you think it could be better, go ahead and apply. You can always retake the test next year and really focus on prepping for it.
• “What do I write in my Statement of Purpose?” Four things: 1) Why you want a PhD so bad that you are singing about it in the hallway, 2) what you will do with a PhD, 3) what specific topics or questions you’re interested in, and 4) why that school’s a great fit.
• “Who should write my recommendation letters?” Ask the best-known researchers you know in the field that you are applying. Next, ask anyone you’ve done research with. Third, ask whoever knows you best and will write these before the deadline.
• “How many schools should I apply to?” Since you’re doing this at the 11th hour, I’d limit yourself to three schools: Your dream school, your “best-fit” school, and a safety school. Otherwise, if you had lots of time, you might apply to as many as 10. (The third time I applied, I applied to 14).
• “If I don’t get in to my dream school will it hurt my chance for next year?” Nope. They either won’t remember you applied (they might have 100+ applications), or they’ll think you’re persistent. And as someone once told me, "The P in Phd stands for 'Persistence.'"
With three weeks to go before the deadline, the most important part of your application is your Statement of Purpose. At this point, you can’t change your GPA, you can’t retake the GRE, and you can’t hang out at the mall hoping to make best friends with a Nobel Prize-winning recommendation letter writer.
What you can do is to write and rewrite your Statement of Purpose. Then have your recommenders give you comments on it. Many students are too shy to ask for this feedback, but it’s the most important thing you can do right now. I didn’t ask for feedback on my Statement the first time I applied, and I got into exactly -- hmmm -- zero PhD programs.
If after reading all of this, you’re still humming “Yo ho, yo ho, a PhD life for me,” take the plunge. Being an academic is a tremendously rich and rewarding calling. Pick three schools, apply, and when you hear back from them, we can talk about course corrections.
Good luck with a great career.
Last night was Halloween. Taking my daughters and their friends Trick or Treating is pretty amusing. Since one of the couples from my Lab didn’t have plans, I asked if they wanted to join us in the 2-hour Viking-like pillage of the neighborhood. The woman showed up as a pink fluffy unicorn and her husband (a graduate student) showed up . . . as me.
This reminded me of grad student days. Even though I didn’t meet much with my advisor, I was really impacted by him. Not so much by how he did research (those days were past), but more by the way he viewed the world and the way he said things to me. Small example: When I was applying for jobs, he asked for a list of schools where I’d like to apply. When I showed up with my Top 30 list, he could have easily said, "Be realistic" or “Are you smoking crack again?” Instead, he said, “Let’s cast the net wider.” A subtle choice of words, but things like this must have made a difference.
Here's why I say that. A couple years after I miraculously graduated, one of my buddies visited me in New Hampshire for a long weekend. While visiting, he asked why I had replaced my broken-down grad school car with a different type that was just about as old. I said, “Because I really like the way it looks.” He then said, “Didn’t your advisor have the exact same car in that exact same color of white.” Why yes, he did. Hmmm . . .
For Junior Faculty, it’s easy to develop tunnel vision that's only focused on your research. By doing so, we can overlook the legacy or long-term impact we will have on advisees, grad students, and teaching assistants. You can leave these students with a real gift, or you can leave them with a lump of coal. They may not remember the details of your theories, but they may remember how hopeful or excited you were about them or about the field. They might not remember any paper you published, but they may remember how you treated them when you were clearly distracted and stressed about something else.
For Grad Students, don't worry about becoming the Mini-Me version of your advisor, but you might be influenced more by them than you realize. Fortunately, you can partly control this. First, you can have some control over who you have as an advisor or mentor. It might take some work to switch away from an abusive or disinterested advisor, but it can be done. Second, you can also choose what it is from those encounters that you want to focus on, repeat to others, and replay over and over in your mind. "Let's cast the new wider" x 100 replays = Good. "Are you smoking crack?" x 100 replays = Bad.
Last night was spooky Halloween. Today is saintly All Saints Day. No advisor or mentor is all spooky or all saintly. Fortunately, you can choose what you focus on. Choose carefully. You might end up with the same car they had.
Two years ago, I had no real idea what a "retracted" journal article was. I had never thought of it, or known anyone who had had one. If someone would have asked me, I would have probably said, "There must be something about it that was wrong."
Two years later, I've had a some journals ask me to rerun analyses and to show original paper-and-pencil surveys from as far back as 2000. When we haven't been able to do so (didn't save the hard-copy surveys or can't track down the coauthor who did the analysis), some of the papers have been retracted because they can't be fully validated.
Since then a few people have asked me if they can believe Article X or Y because they were retracted. I absolutely believe in these articles. The general conclusions are the same, and even if the original hard-copy surveys and original analyses done a number of years ago can't be found, new reanalyses (often done with different statisticians) generate similar conclusions.
On Monday, one of my coauthors pointed out an insightful article that Dr. Stephen Holden had written for The Conversation website titled, "Retraction of a Journal Article Doesn't Make It's Findings False."
One of his many points is that “Retractions may be important signals of reduced confidence in a finding, but they do not prove a finding false. This requires replication.” Social science moves forward because of replication and not rereanalyses.
The Gold Standard of science is when independent researchers can run about the same study and come up with similar conclusions. This is why I wish I had published more detail in the method sections of some of my papers. Instead, when the editor said “Cut out 1000 words,” we cut it out of the methodology section or the references and stayed focused on the idea. Today, you can include all of the methodological detail and post on an online appendix if there isn’t room in the paper.
Looking back, I’d also archive all of the analyses of my coauthors instead of assuming they’ll always have them on their computer or assuming they’ll never be needed. Coauthors can lose laptops, change computers, or leave academia.
As for saving the original pencil-and-paper copy of surveys and coding sheets . . . do so if you have your own warehouse (the photo above was taken in 2005 in Illinois in what was supposed to be my garage). Otherwise you can collect it in a retrievable electronic form (if you think electronic collection won’t affect the results).
Academics is an incredibly enriching calling. As you move forward in a happy career, I hope this is the only time you have to read about a retraction.
A week ago a writer for the Atlantic asked me if I had any advice for young scholars. I have lots and lots of advice, but here’s an expansion on three quick points I shared.
1) Being a scholar and an academic is an unbelievably great calling. It is totally enriching, and you can't let some of the bumpy times that you see me going through dissuade you from a great career.
2) No paper’s perfect, and you need to find the right balance between perfection and completion -- between perfecting something versus completing it. There's tons of "perfect" dissertations that have never been completed. There's tons of "perfect" papers that have never been submitted. You want to balance off polishing the perfect paper for 20 years with trying to make sure that if there were any errors they probably wouldn't change the basic conclusion.
3) You can do research to impress other academics, or you can do research to solve problems. Doing it for academics is more prestigious, but doing it to solve real problems in the real world is very gratifying — very enriching. Having someone say, “I do something differently because of your research, and it works for me” takes away some of the the sting of embarrassing mistakes.
For 30 years I have hoped my research would make people healthier and happier. That's been my specific mission, and it's very motivating. If your research is pointed at a mission that's important to you, it can be incredibly motivating , , , even in bumpy times.
How do you pick the perfect dissertation advisor?
A couple months ago I was at a reception where I met a PhD student from a distant department. Let's call him Jack in the Behavioral Nano Hermeneutics Department. At some point he mentioned his goal was to have his dissertation proposal nailed down by the end of the summer. I asked Jack if he had a committee in mind, and he said, “Oh, yeah, my adviser’s the most famous guy in this field.”
Picking the hottest, most famous person in a field is one way to pick an adviser. After all what could go wrong?
Case Study #1. A number of years ago at a different university, I had a good friend who was starting her PhD in environmental engineering over a second time. Her first go-around had been after she chose the “most famous” person in her field at the most famous school in her field as her adviser. She hated it, hated the school, and ended up leaving with what she called “a consolation Master’s degree.” She said her famous adviser had never around, never cared about her, never thought she was smart enough or working hard enough, never liked her ideas, and that he played favorites with the more advanced students.
Case Study #2. I too had originally chosen the “most famous” person in my field, and things didn’t work out. As a 3rd year PhD student I thought I was going on the job market. Instead I was told my funding was being eliminated, and that I had 4 months to find a new dissertation adviser, a new dissertation topic, and to defend that topic, or I would be asked to leave the program (probably without the consolation Masters).
One conversation rescued me from having to start a PhD a second time a different school. Three shell-shocked days after being blind-sided, I was talking to a friend who was a professor in the medical school. I told him what had happened and about my confusion. He said, “If I knew you were going through this, I would have told you what I tell my graduate students. ‘When it comes to picking a thesis committee, you pick your best friend to be your thesis adviser, your favorite uncle to be one committee member, and your favorite cousin to be your other.’”
This is a radically different approach than what I had used, what the environmental engineer had used, and what Jack was using. The advice was to “Pick your best friend to be your advisor.” Not “the most famous” person in the department. Not even the person whose research interests are most like yours. Pick the person who likes and believes in you and your best interests. You might not be as “hot” when you graduate, but you might be a lot more likely to graduate in the first place.
I’ve been thinking about this because this past weekend I looked up “Jack” to see if he wanted to take a dissertation break come over and meet some of my grad students. On his department’s website, I noticed that he was about the only 3rd year student who wasn’t a formal part of any of the research groups in the Lab of his “famous advisor.” That was like me. Fortunately, I was given a second chance.
Picking a star-spangled dissertation or thesis committee that you think will make you “hot” on the job market is a great strategy for Super-Duperstars. For the other 90% of us, we should pick one that will help us graduate.
How to graduate, get a job, get tenure, solve problems, and make fewer mistakes.