Managers Say the Majority of Information Obtained for Their Work Is Useless (Jan 4, 2007, Accenture)

“Managers spend up to two hours a day searching for information, and more than 50 percent of the information they obtain has no value to them. Nearly three out of five respondents (59 percent) said that as a consequence of poor information distribution, they miss information that might be valuable to their jobs almost every day because it exists somewhere else in the company and they just can not find it. In addition, 42 percent of respondents said they accidentally use the wrong information at least once a week, and 53 percent said that less than half of the information they receive is valuable. 36 percent said there is so much information available that it takes a long time to actually find the right piece of data.”

How Executives Stay Informed: A Study of Resources Used and Time Spent Locating Critical Business Information (Nov 4, 2005, Bersin & Associates)

37% of executives spend more than 4 hours per week searching for information; 36% reported spending 2-4 hours. “Considering the typical salary of a top executive, this translates into $1,000 or more per week in time lost through information searches, to say nothing of the cost of not spending this time running their operations. This estimated figure does not include costs associated with lost opportunities, delayed decisions, or other business impacts.”

I was nervous at the beginning of the defense, mostly because I felt like I wished I had made more progress on firming up the research designs between the pre-proposal meeting on Oct. 3 and the defense on Oct. 25. But, the meeting turned into a really interesting (and fun!) brainstorming session, which I found incredibly valuable. It is almost like the ‘defense’ happened at the closed pre-proposal meeting, and the public meeting was more like a working session. So, even though it added some extra stress to my life over the past two weeks, I’m really glad I ended up doing the public defense — I got a lot out of it. There was a really interesting conversation about changes to my experiment design that will hopefully mean less work for me; I need to think about the potential changes a bit more and talk it over with my advisor next week.

I’m not sure I have any reflections or meta-level thoughts about this stage of the process that might be useful for other people. Thinking back, one thing that helped me a lot was having written up little mini-reviews about chunks of literature that I’d read in the past, and rough sketches of research designs; this made it a LOT easier to put together a rough draft of the proposal document than it would have been if I had to start from a blank page. Also, I spent a lot of time over the summer thinking about my research questions and scope, so when it came down to defending why I chose these particular research questions and how they fit together I had a reasonable argument. (This is an example of the kind of ‘tough’ questions that were asked in the pre-proposal meeting, rather than the public defense.)

I feel like my committee was on the same page about what a dissertation proposal should look like, and at what stage one is “ready” to defend the proposal. In psychology, I think it is more normal for students to propose earlier rather than later, maybe after having collected some pilot data, but before conducting any of the dissertation research. The proposal is then an argument for studying a particular research problem in a particular way, supported by a literature review, and including specific research designs and their expected outcomes.

I’m not sure this model would work for somebody on the “three paper” plan, or someone planning to build something as part of their dissertation. It seems like with the three paper plan, the proposal might be more about how the papers fit together than about what the research will be and how it will be conducted. If the dissertation involves building something, the design of that something will in a sense constrain the output of the work (just like any research design bounds the expected outcomes); if producing the design is part of the research problem, it is less clear to me at what stage the proposal should happen.

anyway. here is the document, including annotations in the margins indicating changes from the last version (yay, latex). off to ASIS&T tomorrow!

• Clarification of terminology in the introduction and lit review sections.
• Improve the problem statement — essentially, more specificity in an early section of the paper about exactly what problem I’m trying to solve, and for whom. Link this with a section before the research designs, and make it more clear how the three studies fit together to help solve the problem
• Some additional lit review on recognition memory, label-following in HCI, and browsing behavior. Also expanding the section on common ground — I’ve done the reading
• Minor changes to the research designs

I’m starting to think about whether a short scenario might serve to illustrate the situations my research addresses. The danger in including a scenario in the proposal is that it may not be the ONLY scenario, and sometimes including a particular one can make it hard for readers to see that other scenarios are possible too.

Part of the difficulty in crafting a scenario is that I believe the problem I’m studying is present in a diverse set of software tools, including blogs and wikis, shared repositories of documents or code (whether using Drupal or CVS or network folders or Sakai), basically any time where you have people putting information out there on a server for others to find and access before it can be used. The user interaction is slightly different with these various software tools, because each system makes certain kinds of tasks and goals easier than others. But the underlying fundamental (psychological) problem is the same. People can effectively communicate with each other face-to-face (for the most part) — why can’t we name files so others can find them?

I’m looking at individual-level behaviors and outcomes, rather than group-level. I often find it difficult to grasp the abstract concept of a group-level behaviors; one can analyze the actions of group members in aggregate to see group-level patterns, or describe qualitative characteristics of a group; but, it still comes down to individuals choosing and behaving in certain ways. In the early days of HCI researchers studied single users at their workstations in isolation, assuming they did not interact with one another; however, interaction among users is central to the situations I am interested in. In fact, defining how that interaction takes place is one way these situations might be classified:

1. collocated group members with face-to-face interactions (project sites)
2. distributed group members who interact via communications media (distributed software teams using CVS)
3. members of a community who use the same shared repository or information resource (the wiki for our school) but don’t necessarily interact with one another
4. users of a public resource who contribute content that others use (del.icio.us)

I think what can be confusing and hard to wrap one’s brain around is the idea that the information producers are also the consumers. On the surface, this seems like one of those “well, duh” concepts, but it has some interesting and important reflexive implications, that are key to my dissertation. In a broadcast situation, like TV or online news or RSS feeds, the producers and consumers are different groups of people, and it is easy to separate the distinct roles. But in a shared repository, the choices and behaviors of users when producing or contributing content determine what can be found and accessed in a repository. The document hierarchy is therefore co-constructed and evolves over time; individual behaviors and choices of one user can constrain the options, choices, and behavior of others. So, the individual level of analysis isn’t exactly clear-cut.

In my dissertation research, I have chosen two approaches. The experiments separate the producing from the consuming, removing the simultaneity of co-construction to assess the impact of common ground on hierarchy formation, and then the impact of different hierarchies on the ability of users to locate documents. The field study, on the other hand, includes data collection about individual behaviors, site characteristics, and group characteristics.

I left the pre-proposal meeting with the impression that a public defense would not be necessary; but we had talked about me doing it regardless of the outcome of the meeting since it is technically a requirement, and our department hasn’t had a public proposal defense for a while. So, come to the defense if you can! It will be nice to have moral support.

I have yet to find time to really sit and think about the feedback I received at that meeting, and the changes I need to make to my document and my research plan. I put off some teaching stuff in favor of the pre-proposal meeting, and had to play a bit of catch-up in that area. But now I need to have a plan for how to make all the changes by the end of next week, before I go to ASIS&T, so that my advisor and committee have time to review the changes before the defense. It all seems a bit overwhelming at the moment, but I’m sure that once I have a plan in place, I will feel better.

i sent out the first draft of my proposal on monday a little after 5pm, in advance of my pre-proposal meeting on wednesday morning. i felt like this was REALLY short notice — i had wanted to send it out by the previous friday at the latest — but teaching stuff got in the way. my committee is really great; they had all looked at the document by this morning’s meeting. i prepared a presentation for the meeting, and we ended up going through most of the slides.

the meeting went really well! everybody seemed really interested both in my topic, and in providing helpful feedback. i am too tired right now to write a meta-analysis of the kinds of things we talked about, but the outcome was very positive. and, i should be hearing back from my committee in a couple of days, after they’ve had more time to read and think about my document, about whether it is necessary for me to do a public defense, or just make some edits to the paper.

hooray!

it is unclear, however, how “final” my research plans need to be at the pre-proposal meeting. if i spend a ton of time perfecting my plans, i run the risk of having wasted all that time if my committee barfs on it. too little detail and i’ll look like i don’t know what i’m doing and i’m not ready. it is a dilemma. it is looking like i will err on the side of too much detail, because that’s the kind of person i am and because i think i will receive detailed feedback in proportion to how much detail i provide.

the scheduling process for this meeting was, i must say, a pain in the ass. i probably didn’t go about it the right way either. i looked at my advisor’s schedule, which i can access online, and then emailed out a list of times that worked both for her and myself to the rest of my committee. well, it turned out that none of those times worked for more than two of them. total failure for my first attempt! at that point i got creative, looking outside the timeframe i was comfortable with… asking my advisor if it was possible to reschedule meetings… sending out suggestions to everybody only to be rejected and have to come up with a different set of suggestions… the entire process took over a week and dozens of emails!! amazing.

the good news is, it is scheduled. for a week from tomorrow. yeah!!! it is good to have a deadline. i am so mentally and emotionally ready to get this overwith and move on to the next stage. but, i’m not sure the work is entirely ready yet! and while my stress level about teaching this term is going down now that we’re moving into material i kinda sorta actually know, the amount of time spent planning has not decreased at the same rate. the next six days will be really exciting. (not really. i’ll just be writing.)

if you’re interested in an overview of my dissertation proposal as it stands right now, you can take a look at the poster i created last week. i will be sure to post about the pre-proposal meeting after it happens.

i used to love fall. when i was a little nerdy kid by the time August rolled around i was so ready to go back to school, i couldn’t WAIT for fall. for the past few years though, September has mostly meant sleep deprivation. with getting back into the teaching groove and the CHI deadline every year, i don’t look forward to fall as much as i used to.

and i can’t slow down yet! the annual phd student poster session is friday, and i haven’t even thought about making a poster yet. i am also in the process of scheduling my pre-proposal meeting and proposal defense, both to take place before November. ahhh, grad school.

- If you’re planning to have a pre-proposal meeting with your committee before the proposal defense, don’t spend a ton of time writing the proposal document in advance of that meeting. Save the effort for after you’ve received feedback from the committee.

- It isn’t necessary to analyze the heck out of a dataset right from the very beginning. Prioritize the analyses in such a way that you answer the most important or theoretically/practically relevant and interesting question(s) first. You can always go back and slice the data a different way later.

The second bit has helped me focus my analysis of the Taboo data. I’m doing a kind of conversation analysis, and I put together a fairly long list of things I could code for.

But, after looking at some of the data with my advisor, we decided that there are really two things we want to know right away. The independent variables in this study are “shared past experience” common ground and gender; I recruited 32 same-gender pairs of participants, half of whom were friends and half were strangers.

1. Does common ground allow people to communicate more efficiently and converge more quickly on the target word?
2. Do people who have common ground take advantage of it when forming their referring expressions?

The objective is to compare ‘referring expressions’ under different common ground conditions. Each clue and guess can be considered a referring expression. I’m also interested in utterances that play a feedback role, because some people might be better at providing feedback than others (and feedback is an important part of social language use).

So with this in mind, my first pass through the data will be coding utterances into clue, guess, or feedback. I will also identify instances where common ground is used. Each word for each pair will be coded as success, fail, pass (if the participants passed on the word), or error (if the clue-giver used one of the illegal words). All of this should only take me a couple of days, since the data is already cleaned! Hooray!

- A dissertation that consists of multiple studies should have one study that is the ‘main’ part of the dissertation, that makes up the bulk of the work. It is ok to have pilot data or preliminary data collection, but it is important that the dissertation not end up looking like the accumulation of a couple smaller, conference-paper-type studies.

- The Taboo study is interesting and definitely relevant, but it might be better not to include it as part of the dissertation — it might confuse people because it was collected at a different time and for a different purpose.

- I have set a rather aggressive timetable for myself; I should think about defending around February 2009 rather than doing it in the Fall term of 2008. That way I can concentrate on getting a couple of papers from my dissertation work out the door, which will (hopefully) make me a stronger job candidate.

- Research topics tend to come and go in cycles, and at this point there don’t seem to be a lot of people in HCI or LIS looking at the same thing I am — but my topic area seems to be gaining momentum again, so I have a good opportunity to make an impression with my work.

I think it will be interesting to look back on these notes at the end of the process — I am guessing that they’ll be more meaningful in hindsight than they are right now.