Waay too long!! I put faaaar to much onto my slides and didn’t even get through half of them. 10 mins is so fast- I need to refine, summarise and streamline more. I had no time to practice and panicked about putting everything I had done on the slides so I could get some good feedback. I had gotten too immersed in the data and forgotten to summarise.
However it was such good preparation for the real thing and really highlighted where I have to narrow things down and what to include on my workflow instead. Succinct summaries are the way forward- and I need to look at how I can put everything across in a simple and effective way.
It also highlighted I had achieved a lot and done a lot in the time so far, so I need to be kinder to myself about feeling like I have not done enough.
I will investigate ‘lightening talks’ as Vikki mentioned those ( a terminology I was not aware of) and see what I can learn from the experts who fit huge projects into short talks!
This was an odd experience for me, I had lost my voice so was not able to speak any contributions to the workshop…only type! This meant I had to be very succinct about what I said but also realised how long it takes me to type things out. It also made me reflect on how much I was able to actually contribute when only typing. Something to think about for future online sessions with students…
I found this session really useful. I hadn’t started any of my data analysis yet. I had the data in raw interview form but had not even transcribed it yet. It was really fun to try out different methodologies of analysis without the pressure of using your own data set,how they work in practice and question my understandings of them.
We started by practically looking at semiotic analysis- the study of signs and symbols and their use or interpretation.We looked at images associated with ideas of a researcher.
It was really interesting to see the groups assumptions on what a researcher should be/do. Mine have defiantly altered since undertaking this SIP. I come from an art and a science background so sometimes there is synergy but other times the different schools of thought do not align. The last time I carried out research it was science based. My science head is more rigid and facts based whereas my art head wants to explore experience. Hopefully my project will capture both and my two heads wont contradict each other or be counter productive!
Data poetry was an interesting exercise. I liked the limitations of only using words there and having to use the order of speech. It was amazing to see how different people’s interpretations/stances/takes on reading the same thing, and what they chose to illuminate or bring to attention. It really highlighted to me how the researcher is so pivotal in how data is analysed and presented back.
Searching for themes was another really useful exercise. It is an area of analysis which I had already identified as something I would probably do with my data.
We were asked to read something and analyse our
assumptions about the data and positioning
initial observations, responses, insights
reflect on assumptions
then code transcripts
This was interesting to me as it thew up some of the things I was thinking about with my analysis of my transcripts and how I might interpret things. It also gave me a good idea of how to approach thematic analysis in general.
I found the exercise of breaking everything down into 5 sentences very difficult! I know this is something I will struggle with- streamlining what I need to say about my project. I’m so invested and in the thick of it I worry about the balance of saying enough and getting the points across in a way that someone who is new to the project will grasp with ease.
After the session I knew I needed to research thematic analysis further. I listened to the talk by Braun and Clarke about their new book. It was very informative, but it did send me down a further thematic analysis rabbit hole about the nuances between different types of thematic analysis, how they are used and when to use them.
I found this page quite helpful in breaking it down
I was initially hesitant to put together another presentation for this session but I ended up doing it and I’m glad I did. Vikki suggested timing the 10 mins so that we get used to the time for our final presentation. This was a really useful exercise, I talked through my slides but didn’t go into vast detail, I was a little under time which was good but by the time I finish my project I will (hopefully) have so much more to say it made me start to think about how to present my work for the final presentation. I get nervous on the spot for asessment and I don’t love powerpoints….is there another way I can present my findings? Make a little animation? Voice Record ahead of time so I can play a video on the day? I need to find a more creative way to do this that sits better with me!
I did listen to a Helen Kara talk about creative research methods.
Helen described some PHD students who used interpretive dance to present thier findings! On further googling there is a Dance Your PHD competition for the sciences which I had no idea about!
While I am not a dancer and will not have time to create something amazing like this, my brain is ticking over thinking about things I can or might be able to do….but that’s jumping ahead a bit! I will also read some sections on Helen’s book around data prep, de-coding and presenting findings.
It was really useful to hear where everyone is at with their projects and how they are tackling them. There is some synergy of topics within the group which is really interesting, but different approaches.
Something interesting that came up was sampling and how to go about selecting your participants. Each of us were at different stages in the process. One had already sampled and interviewed, I had sampled but was yet to interview and another was still thinking about how to sample and who to contact. The topic of student reps came up which was really interesting and pertinent for us all. We discussed the perceptions around using them as interview subjects – most likely open students comfortable with speaking. However it was brought up that in my colleagues study that the student reps kept referring to and speaking for the general student experience or other students experience. Speaking to others experience is what we want to avoid in our studies. A way to tackle this could be to re-center the conversation to the student in questions experience and remind them that it is only their point of view that you need, and also to omit any information from the study that is anything but the interviewees direct experiences. Perhaps it is not wise to use student reps in a study because of this, perhaps it would be interesting to talk to students who do not normally speak out. It would be interesting to see how different students answer the questions…
For the purposes of my study I had already chosen student reps from all 3 year 3 undergraduate courses in Performance: 3DFX, Hair Make-up and Prosthetics and Costume. 6 people in total to contact. If I had not narrowed it down to just student reps I would have had to contact around 150 year 3 students. I was trying to make the project smaller and more manageable. In an ideal world I would have had time to speak to lots of students but that’s probably more suited to a larger scale research project in the future. My SIP has the potential to be a catalyst for that. I chose year 3 students as they have the most experience with assessment. Yr1 is yet to be assessed and Yr 2 has only been assessed on a pass/fail criteria during the pandemic with limited ‘normal’ technical delivery. I do not know 4 of the students. Only one student replied and I had planned to choose the one who replied first. It turned out to be a student I teach, I had perhaps naively thought that all the students would be interested in participating due to the incentive but only the 1 student that knew me replied. This probably is due to the fact that they know me, but I have never marked their work nor will I so there’s no motivating factor like that for them to talk to me. I will have to re-enforce my position as researcher at the beginning of the conversation. On the plus side we already have an established rapport so I’m hopeful that they will feel comfortable speaking openly to me.
For Technical staff I decided to interview a Technician that has a similar role to my colleagues and I teaching in the Performance realm, but at a different site. I decided to do this as my colleagues and I in my team are so close knit and we have discussed my project and I have asked for their feedback at various points so I wanted a completely different insight, but still within the context of Performance.
For Academic staff I decided to pick a pool of 6 academic staff in the department whom I know but do not work closely with on a day to day basis. I have yet to receive a response from them. I hope they do respond as I really want an academic perspective. If they do not respond I will resort to interviewing someone I know well as I feel its important to still have an academic voice present in my study.
I also wanted to capture the perspective of an alumni. This is because they would have had time to process their university experience since leaving, and their reflections now may be different to the expectations they had whilst studying. It just so happened that 3 are starting to work within our department coinciding nicely with my research project, so I will interview one of them. Again I will do so on the basis of whoever replies to me first to keep it consistent. I used to teach all three of them (but they were on different courses) but hopefully at least one of them will be interested. We are peers now so hopefully there will be limited power dynamics.
As I was feeling a bit lost on the last session, and wasn’t ready to test anything out in the assigned time in our workshop, I organised with a fellow Technician and PGCerter to test out some interview questions I had been fleshing out.
I got some really useful feedback from Vikki just before my interview which helped me hone down my questions more. I had too many and some of them were probably not directly related to the question enough so I axed a few and tightened up my questions.
It was sooooo good to do a dry run of my questions in an interview scenario, without the pressure of ‘doing it for real’. We had a really interesting back and forth and I was conscious to practice my active listening techniques. I put my phone out of sight and removed as many distractions as possible to focus fully on our conversation. I did find it hard towards the end when I finally looked at the time and realised I had a meeting to go to shortly, this distracted me somewhat! I need to make sure there’s plenty of time either side the interview so that this doesn’t happen in future. I also found out that there is a new feature on teams that transcribes meetings automatically which is a new feature I did not know about!
EDIT: I’ve gone back in and while this feature it good, I will still have to go back through it all and make it into something cohesive. I have only ever used this feature on panopto before, and the teams version seems to be even more partial to picking up strange words and making different meanings from what is said!
Some recall happened during the conversation which was really encouraging, at first the interviewee had said they had never been involved in marking before and then a few questions later and a few more probes, mid-interview it was revealed that they had infact done a marking excercise and they were able to comment on that experience.
Bearing this in mind, and the fact that I had not given my interviewee the questions beforehand I decided that going forward I will disclose my questions before the interview session in order for the subject to have the time and space for recollection….which might mean less probing and more time for the interviewee to speak and think freely. Reflecting on this I also decided to invite participants to think of an experience if possible prior to coming to the interview, or to bring an artefact to talk about.
UPDATE: This was used as part of my email to participants:
Before the interview:
It would be useful if you could please find an artefact to bring to the interview or think of an event to describe your experience of technical assessment. This artefact or experience will be used to start the interview exploration. An artefact could be any image or object, learning resource, an outcome from a lesson or workshop you delivered or attended.
The technical academic relationship was spoken about a lot without directly asking about it (this was one of the questions I had omitted earlier) so I found it interesting that this was the natural progression of the conversation. However I do wonder if it’s because the interviewee had heard the ‘behind the scenes’ conversations about my project during the workshops. This will become clearer with the individuals I interview who have not had a backstage pass.
I had a total light bulb moment towards the end of our conversation. It flowed on from the question about what technical involvement could look like. The interviewee highlighted the benefit it would have to them as a marker seeing the student perspective on what they had taught- their reflection of what they have learned from you and your window into that. This was something I had not considered- I had never thought about being involved in the marking process as a way to improve your own practice, and to use it as a useful tool to challenge your perceptions of knowledge exchange that has taken place. In my mind I was more focused on perhaps the potential benefits for the students that this was a really interesting unexpected perspective. I really hope my other interviews provide we with more unexpected insights.
For preparation for my interviews I read the chapter on Semi-Structured interviews. It covers extremely useful and practical information about the interview process and considerations for conduct before during and after. Again, the theme of simplicity strikes me about this process. Do not over complicate things.
Cousin suggests creating a set of themes which serve as a guide to facilitate interview talk which can be used to glean more meaning from experiences.
‘Meaning..is actively and communicatively assembled in the interview encounter’
Both parties must work together to construct meaning and form understandings. A talented interviewer will facilitate a dialogic reflective journey to mutually happen.
A huge important take away is knowing how to listen-practicing active listening. Getting rid of distractions, focusing on words, pauses, and their meaning. Not simply just waiting for your turn to talk.
Use less direct to the point questions, they need to be questions that are more than information seeking. They need to get the subject talking, and they need to be open ended and not be leading in any way.
LESS IS MORE- a maximum of 45min interview
Think of it as a ‘conversation with purpose’ (Burgess 1988, Cousins 2008)
Main questions -> then probes -> then follow up questions
5-8 main questions on the topic is advised. Vikki gave me feedback that 5 is plenty but good to have back up if need be.
Question Styles
Tour questions are good to use- they take you on a journey of an experience. For example I could say something like: I am interested in your perception of how work is assessed. Can you take me through your last assessment?
Stand out events probe for things that have big meaning- Have you ever received a grade or feedback which became a turning point for how you saw future achievement?
Hypothetical situations you can create imaginary scenarios to glean information- If you were in charge, what would assessment look like?
Compare or Contrast e.g. can you tell me about your experiences of technical teaching and academic teaching?
Task Question e.g. can you look at these words and tell me what they mean to you with regards to assessment?
Probes
That’s very interesting can you tell me more about…
Do you have an example of when that happened?
What happened following that?
How do you see that as related to the question?
Follow Up Questions
Get more detail, extra info, illustrations, explanations, exceptions to the rule
Check understanding/ Theorising with the interviewee
Reflect back – it sounds like you are saying….is that correct?
So you feel that ….?
From what you are saying it seems like….. does that sound right?
Picking up on hesitancies, tensions or contradictions
Discover what lies beneath them..
Can you say a bit more about that?
Can you talk me through what happens when….?
Give interviewees the chance to think about difficult questions and then get back to you.
Ending
Checking for missed opportunities- Is there anything you would like to add about potential benefits of technicians being involved in assessment that hasn’t been covered yet?
Thank them and specify they will get a transcript and a date to get comments back.
Field Notes
Reflect on questions and answers
Any unfilled gaps?
Feelings about the setting interviews etc.
How were meanings produced? What might the meanings tell us about the topic in question?
Additional things to consider
How might the circumstances of the interview be shaping what the interviewee is saying? (Intersectionality around gender, class, ethnicity, disability etc) Is there a power imbalance/ asymmetry between discloser and disclosee?
Minimising the power present
Disclose own relevant experiences and facilitate an ‘exploratory thrust’ rather than information prospecting. I understand how this could work but also struggle where to draw the line with this as an interviewee…if you share too much will it steer or influence responses if you do this? Where does this fit with not interrupting the flow or staying quiet to let the subject freely explore?
Maintaining a reflective stance throughout
Keep referring to positionality throughout, what parts of it are problematic? Have a conversational style and build trust
Build Rapport
Quality of conversation is largely dependant on rapport building capacity- this means it could be a good idea to use people I have a working relationship with and not a complete stranger.
Embellishing responses or pleasing behaviour
How can the interviewer be sure to get candid answers? And not obligingly offering what they think you might need? An exploratory dialogic direction to the interview may help to avoid this.
Giving Voice
Giving space and attention to the interviewee, creating a developmental dialogue but manage it so it does not end up becoming like a therapy session. Again, this stresses active listening, letting the participant do most of the talking.
Non-linguistic communication
Body language
Tone of Voice
Pauses and silences
Modes of dress
Pauses and silences have come up a lot in the videos and articles I have read. They suggest leaving a silent space to prompt the subject to continue speaking a little more, as this is often the time the interviewee often will provide unsolicited information with deep meaning. Tone of voice is something I hadn’t really considered. I’m so glad I will be recording the interview and will be able to pick up on these sorts of things- as there’s a lot to think about and I don’t want it to interfere with my active listening at the time of the interview. Having said that I need to listen out for places where I need to probe further, and tone of voice may be an indicator of this. Body language is one to consider in an in-person interview. This is trickier online as you generally only see head and shoulders. (I am giving my participants the ability to choose whichever suits them better) but I suppose I can look out for facial expression. Modes of dress…not to sure what to think about that….seems like that could be tied up in a lot of assumptions.
Consent
Other than explicit consent form which consider research ethics and permissions, Cousin suggests letting interviewees see transcripts and comment on analysis. I will build time to do this within my study.
IS IT A TRUSTWORTHY REPORT– checklist- These are all things I must consider when analysing my data.
Have you avoided smoothing your interpretation? Oversimplifying or too sensitive to variation of experience and viewpoint?
What has determined your decision about the numbers of interviews, and can you defend this? Yes.. it is a small scale research, where time is very limited. It is also just a starting point so I need to keep the numbers manageable.
Have you shared emerging ideas with interviewees throughout and post interview?
Did you build rapport? I will be interviewing colleagues that I have some familiarity with so there will be some rapport established from the get go. At the beginning of the interview we will spend time catching up and relaxing into it.
Have you asked a colleague to check transcripts to explore rival interpretations?
Have you kept a diary to capture reflection and theoretical leads? I am reflecting on my experiences and reading through my SIP blog.
Have you avoided cherry picking quotes?
Have you displayed questions as well as responses to show developmental dialogue?
Did you explore alternative data sources to strengthen your interpretations? (I’m not sure I will have time to do this but I can talk about future thinking, and where I would move the research on to..)
THINK LIKE A JOURNALIST- Kelsey Samuels
This Ted talk grabbed my attention when researching interviews. Journalists spend a lot of time gleaning information and re-telling it. It was a really interesting take on thinking about your positionality when re-telling stories. ‘What if I am wrong?’ What I write makes and shapes peoples perspectives. Evaluate what we hear before we pass it on. Adjust beliefs to fit the evidence in front of us instead of using it to re-enforce personal biases.
During my action research interviews these are all things I must consider and be aware of. I need to be self-reflective and keep checking in if the things that I am saying are grounded in truth and not motivated by my perceived intentions or wants for my project. I am interested in perspectives other than my own so I can look at the question in a different way- are there things I didn’t consider? I’m interested in that as much as the things that I think might come up.
It was so great to have time to chat about our projects with our peers. Its really comforting to talk to people in the same situation and hear their ideas and takes on really interesting ares of study. I find it really inspiring and always want the opportunity to do it more. I think its important to have insights into other departments and ways of working. You can get tunnel vision surrounded by the same team all the time. It is refreshing to get other multi disciplinary perspectives. It has highlighted the value and importance of sharing ideas and knowledge, and how much I crave meaningful focused conversation. I’m really thankful for the space and to practice my active listening skills in preparation for my research.
First I had time to chat to Jo, who is also a technician on the performance courses at Wimbledon. That was really interesting as there were many parallels to our work. We only had 10mins to talk about what part of the action research cycle we were at. The conversation was so interesting, were currently trying to find time to have another chat.
In groups again we talked about methodologies and what we were using. We focused of focus groups and talked about the difficulties of being a neutral facilitator. We discussed the options of moderating yourself or standing back and letting someone else do it. We also talked about knowing when to step in to stimulate conversation and knowing when to just let the conversation flow. We talked about our inexperience in facilitating focus groups and how that may impact the quality of the conversations that are had. We talked about online/in person dynamics. We talked about the lack of control and over the way the focus group would go but maybe unexpected turns would be good for our research. Lastly we discussed the pitfalls of interpreting the conversations post focus group, are you giving a true representation of what occurred or are you trying to fit it to your research?
The last group exercise was discussing projects more in depth and using it as a testing area for feedback. Rachel used us as test interview subjects for her project on citation. It a really good exercise for us both. It was so useful for me to be the interview subject and see how it felt. Rachel used a fun way to break the ice- a little game to find out which citation character you were. It was a really great way to get started and had me thinking about ways I could inject a bit more fun into my project. In fact Rachel suggested using an ice breaker in my focus group as part of my feedback, to help relax the room and break down the power dynamics- something I hadn’t considered yet, so that was really useful! The exchange also got me thinking more about my positionality as a researcher and the questions I was asking.
Ching Li talked through her project around imposter syndrome. We talked about how she was presenting the work and showing a video and wondered if it needed both. We thought through narrowing the project down to feelings of imposter syndrome within an educational setting to avoid becoming too personal. We asked if she wanted to inform students of what imposter syndrome was or find out what they know about it and what they understood about their experience?
I talked over my project but was still feeling very unsure how to progress with mine at this point. This was before my chat with Vikki. It would have been so useful to use that time to test my interview questions but unfortunately I was not that far forward yet. I am going to orgainse testing my questions as I have seen first hand how important that is.
At this point I feel like i’m searching for one thread, when there are so many interwoven ones.
I schedule a meeting with Vikki as I am just not feeling creative, have such little time due to work commitments and caring responsibilities and am really struggling with how to move forward.
After the chat the fog started to clear a little.
Vikki reminded me that while its good to be invested in my project I need to practice self care and do what is manageable. I’m overstretched. Keep it simple. Make life easy.
Interviews- I’ve already been pretty much carrying them out and have got some good information already. We agree this is a good way forward. Forget the complexities of a focus group and survey. Interview a few people, focus on 2-3 key articles.
Re-frame the question. Steer away from questions that necessitate a yes/no answer. Its an inquiry. You might not find a solution. Make it more experiential- i.e what is the experience? How can I understand it?
A spotlight on assessment practices in performance:What are the potential benefits of technicians’ input into the marking process?
Questions that are flying round my head and I need to refine more and get answers to.
brainstorming questions at the very beginningmore initial brainstorming questions
I started to refine the questions after more thinking and research. Yes/no answers are not a good idea even when followed by open ended questions. Research suggests people are more inclined to select yes, even if the answer is no (Berg and Rapaport 1954)
So apparently this mythical beast does exist. You can still be a teaching and learning technician and not just purely management at grade 5. My manager was telling me about a job role advertised at CSM that had lots of technical managers up in arms because usually unless you transition to solely management and not teaching, you cannot become a Grade 5 technician. Therefore there is a ceiling to career progression. However this new job role was created because there is a demand for industry professional technicians in education (digital) who simply don’t want a job that is paid less than what industry does, so they were not attracting any applicants. This new role may encompass marking depending on the course. For example courses that are becoming more inter disciplinary and technicians are being brought in to teach students specialised skills. Skills that markers and course leaders do not hold. How can they be marking work without consultation from said specialists? Sometimes this is requested from grade 4 technicians, but where, as technicians, do we draw the line if we are not remunerated for the work?
This got me thinking about expectations and job roles, and the apparent lack of transparency between job roles. There is a lack of understanding inter departmentally what every one does, where their expertise lie, what they are interested in and what sort of research (if any) that they do. Technical job specifications are more and more competitive, Universities are asking for more and more from technical staff, a degree, a postgraduate qualification, industry experience, teaching experience..the list goes on. Yet if I look at the department as a whole, technicians are the staff who have the most PGCerts in comparison to the academics. Some course leaders have HNDs and some have PHDs, some technicians have MA’s or MSc’s. There is a large variety in qualifications. Yet technicians are perceived as lower or less than academics, and are put on pay scales to reflect that. Add to that the Visiting Lecturers who mark students work who have only industry experience and limited teaching experience, but are paid significantly higher rates, and it all feels a little bit uncomfortable.
If a technician was to be included in the marking process, where would they sit on the scale? There seems to be an emergence of the technical/academic hybrid. I am one of them. Yet for the majority of the time I sit on the lower scale, only paid extra for small bits and pieces of work. The University demands these new forms of institutional spaces because of the way teaching has evolved, yet the hierarchy still exists. There has been little reform there- we still come under ‘support’ staff under the same guise as non teaching, non student facing staff. It seems that the language needs to be changed as well as career progression, especially within the art school where technical/academic roles are becoming increasingly crossed over.
In the article Lets bridge the gap between lecturers and technical instructors in the guardian from 2016 touches on a lot of the things I have been saying and feeling. Its from 2016 yet nothing has changed! I read it and its still relevant today.
‘Technical’ is not a dirty word in industry and it should not be in higher education. Having two disjointed teaching teams is unhelpful and creates another political minefield that the students have to navigate.
Guardian, Academics Anonymous, Lets bridge the gap between lecturers and technical instructors
Is this the way forward? Change the disjointed nature of the hierarchy within teaching teams…transform it to a community of practice where the language and pay reflects job roles? I cannot change the assessment process or the structure of the university overnight but I can probe and question the way systems work, and how students and staff experience these systems.
While both the chats I had were really great I am feeling increasingly lost….
Should I do a focus group with all participants? My main concern is that students won’t feel free enough to say what they want to with staff present (like course leaders). But I am interested to find out what happens when everyone is in the room. Its a tough call. What artefact do I use to get everyone talking? If its a marked piece of students work and there is a marker present (who marked said work) that will be an odd dynamic…how would that work? What other artefact could I then use?
Or should I start with a focus group with just the students and then use my findings to inform the staff survey?
I am also struggling with feeling de-motivated. The course I primarily work on has just been paused, will not recruit next year and will be having a huge overhaul. Its not that I don’t want change, the way it was handled was not transparent and made us feel very under-valued, and that our opinions don’t matter.
So with all this floating around my head I’m finding it difficult to focus, pin point, hone questions, be decisive.
The more I read about un-grading the less sense the marking process makes to me. I suppose it supports my feelings on putting more focus on the learning journey- the process, the trial and error, rather than the final outcome. It highlights the value of some of the things I am thinking about- emphasising the entire portfolio for example- looking at the whole journey holistically- activities, reflection, writing and wondering…this all needs to be taken into account- the ENTIRE experience. With cut downs in the number of pages digitally submitted for assessment, and only academics involved in the marking process, what message are we sending students about what type of work is valued?
Having students develop an individual plan- this feeds into what I am thinking about the technical journal/diary/log bigger picture. And conducting portfolio conferences- this is something that’s done, but in my opinion, not enough (only at the start and end), and not enough with technicians involved too.
This quotes gives me hope:
There is nothing ideologically neutral about grades, and there is nothing ideologically neutral about the idea that we can neatly and tidily do away with grades. We can’t simply take away grades without re-examining all of our pedagogical approaches, and this work looks different for each teacher, in each context, and with each group of students.
The relationship between students and teachers suffers when our systems and policies reinforce hierarchies and encode biases.
Jesse Stommel, Grades are Dehumanising: Ungrading is no simple solution
I think I’m getting bogged down by all the hierarchies that are reinforced with marking. And technicians not being included is such a big part of that. I suppose in a way by adding in the technical voice as a champion for the students, I am in a way trying to humanise the process more. Question the location of knowledge in the room…
I need to remember its ok to explore things, its ok not to have all the answers. I can think what if, even if its messy and difficult.
