Preliminary Analysis of Speed & Ease

After the testing transcription methods trials in the Fall, the research team performed a preliminary analysis of the findings, in particular the speed and ease of use for each of the transcription methods: ExpressScribe, SpeechNotes, and Stream.

Speed

We tabulated the overall average speed of each transcription method over the course of the four interview audios by each research assistant (RA). Certain RAs (such as student E) took more time than others who were quicker across the board (such as student C), which is quite normal and to be expected. (See below.)

As you can imagine, speeds also varied based on the number of the trial (as they got more familiar with the methods), by order (which was done last and they were more familiar with the transcript), and the audio file (as some were quieter or had more background noise. This is a preliminary analysis, so we’ll be digging into those details more later.

But, when taken overall, the average time per audio file was greater for SpeechNotes at 52.6 minutes and comparable for Stream and ExpressScribe at 45.2 and 45.3 respectively. (See below.) We found this a bit surprising. However, three students had previously worked with ExpressScribe, and they had practice with this. This familiarity may have impacted the speech. Additionally, ExpressScribe is a single program and one process for timestamps, editing, etc., whereas Stream is a multistep process.

Ease of Use

1. All students preferred ExpressScribe.

In terms of perceived ease of use, across the board, all students preferred ExpressScribe. While surprising, we again note that three students had previous experience with ExpressScribe.

Additionally, only 2 students had done a full hour length transcription, and working with a two minute transcript is very different than the tedious process of working with longer stretches of transcription. Since these are only 2 minute audio files, the students may not have had a accurate picture of what it is like to sit for hours transcribing 40+ hours of data with ExpressScribe. So based on this limited experience, they may have thought the steps weren't worthwhile, while someone who had transcribed substantially might have a greater appreciation for the assistance.

Additionally, one student who had transcribed substantial data had never used any software for that process. They just typed into a Word document and listened to the audio, so to them this was very advanced technology. 

2. Students unanimously disliked Speechnotes.

On the other hand, student unanimously disliked SpeechNotes. This coincides with the fact that it took longer, but students also reported that for some of them it didn't’t pick up audio from their device or another device so they needed to repeat or revoice it. Even then, some had trouble. One student mentioned that the issues might have been caused since Spanish was their L2. (And this is of course another factor to consider here.)

Finally, students reported that they liked Stream more with practice. Much of this might have been due to experimenting with the editor in Stream versus editing in the downloaded file as well as the multistep process of creating a video, uploading it, running the script).

3. Student like Stream more over time.

Finally, students reported that they liked Stream more with practice. Much of this might have been due to experimenting with the editor in Stream versus editing in the downloaded file as well as the multistep process of creating a video, uploading it, running the script).

Next Steps

Moving forward, in Spring 2021, we will be holding Corpus Internship classes at both UTRGV and UA. As a part of this grant and project, we will analyze ease of use and accuracy for these methods with the students in the classes.

Due to these results, we’re thinking about asking students to transcribe a full hour for each method, or at least a longer audio segment. We think this may help with the effects of initial training as well as students not having substantial experience with manual transcription.

We also plan to have RAs or our research team complete certain steps, such as creating all the videos, uploading them to Stream, and downloading the transcripts in Stream. The team can run an initial script which will delete some of the extraneous lines to ease the transcription process for students. Then, the team can run the second script to collapse the lines so that it shows per speaker. 

We will also be working with Ms. Jessica Draper to revise the script further so that it can break these lines with multiple turns into separate lines automatically. She will re-work the script in R so that it can run as a program over multiple files within a folder at the same time. We will share this script and process in detail once finalized on the blog.

As we move forward with this process, we will continue documenting all of this on the CoBiVa blog in future blog posts.