Skip to content

Transcriber

Jump to bottom
Oloff Biermann edited this page Dec 17, 2019 · 10 revisions

For Transcriber implementation, we made use of the Azure Cognitive Services .Net SDK.

Reason to Choose Azure Cognitive Services

  • It was a recommendation by HSBC.
  • It has language specific SDKs instead of only providing APIs. This allowed us to call methods native to whichever language we would use.
  • At the time of initializing this project, we compared multiple Intelligent API products and narrowed down to Azure Cognitive Services and Google Speech to Text API. We found Azure Cognitive Services is more mature and more supported functionalities than Google Speech to Text API. And based on our selection, we also picked C# as our implementation language to be better supported by the Microsoft Windows environment. Mid-way during the project Google API came out with a beta release of a revamped library, but we continued with Azure because we decided only when Azure did not work out would we switch to Google. And Azure did work out fine.

Method used in Transcription

From the Azure Cognitive Services, for transcribing, we tried out different mechanisms Microsoft supports: pull or push, and different data format Microsoft supports: Final or Stream or Binary stream, and found Pull stream is most accurate and used it.

Also, in order for the voice identifier to determine when a new person starts talking, we tracked time offsets in milliseconds at the beginning and end of each text segment detected by the Azure Cognitive Services.

Detailed Method Readme

DiScribe.Transcriber.SpeechTranscriber

Provides transcription of meeting audio to produce a set of transcription outputs. Uses the Microsoft Azure Cognitive Services Speech SDK to perform transcription.

  • DoTranscription() Creates a set of TranscriptionOutput objects containing speaker names, timestamps, and the contents of what each speaker said. The transcription follows the the correct order, as the returned collection is a sorted list which is sorted by start offset.

     

DiScribe.Transcriber.Recognizer

Provides speaker recognition functionality via the Azure Speaker Recognition API for up to 10 speakers that are known to this instance.

  • Recognizer(controller: TranscribeController) Creates a new instance of Recognizer using the specified TranscribeController
    • controller- Provide access to DiScribe.DatabaseManager.Data.User objects which represent potential users to recognize

 

  • DoSpeakerRecognition(transcriptionOutputs: SortedList<long, TranscriptionOutput>, apiDelayInterval: int = 3000) Recognizes a speaker from among up to 10 speakers within each TranscriptionOutput instance in TranscriptionOutputs. Potential speakers are the set of DiScribe.DatabaseManager.Data.User instances accessed from the TranscribeController. For each TranscriptionOutput in transcriptionOutputs, this method considers the audio segment associated with that instance. Matching is by the Azure Speaker Recognition profile GUID. If a match occurs, sets the Speaker property in each TranscriptionOutput to represent the recognized speaker. Else, if no match is found, the TranscriptionOutput instance is marked as unrecognized.
    • transcriptionOutputs- A sorted list of n transcription results with audio segments and transcribed text
    • (optional) apiDelayInterval- the time between successive API requests in ms

DiScribe.Transcriber.TranscribeController

**Presents an interface to the speaker-recognition based transcription functionality. Allows the use of a set of Voiceprints to perform transcription of a meeting audio file. Also supports emailing of a transcription file. **

  • TranscribeController(meetingRecording: FileInfo , voiceprints: List<User>, speechConfig:SpeechConfig, speakerIDSubKey:string = "b140a43050f24bef910532e6628b3484") Creates a TranscribeController instance using the specified meeting audio, list of users present in the audio, Azure Speech Recognition API configuration, and Azure Speaker Recognition API key

  • Perform(lineLength:int = 120) Uses Voiceprints to perform speaker recognition while transcribing the audio file MeetingRecording. Creates a formatted text output file holding the transcription which is accessible through MeetingMinutesFile property.

  • WriteTranscriptionFile(rid: string = "") Writes transcription specified in MeetingMinutesFile property.

Clone this wiki locally