In final part of my multipart article on converting my Murdered by Midnight Board Game to a Mobile Game App using Swift and SwiftUI, I’ll cover supporting Music, Sound Effects & Voice in my Multiplatform SwiftUI game.

Hopefully, other indie developers can find this information useful.

Index:

  • Part 1 – Deals with converting a mockup to SwiftUI, handling different device sizes & screen ratios and displays screens as needed.
  • Part 2 – Covers adding Game Center support to a SwiftUI app.
  • Part 3 – Covers handling tvOS quirks in a SwiftUI app.
  • Part 4 – Covers handling macOS Catalyst quirks in a SwiftUI app.
  • Bonus Round – Covers handling Music, Sound Effects & Voice in a SwiftUI app.

Handling Background Music and Sound Effects

I knew I want to have background music in my game, along with the ability to play a few sound effects at the same time, over this music. Here is the common class that I cooked up to handle everything:

import Foundation
import AVFoundation
import SwiftUI

/// Class to handle playing background music and sound effects throughout the game.
class SoundManager: NSObject, AVAudioPlayerDelegate {
    typealias FinishedPlaying = () -> Void
    
    // MARK: - Enumerations
    /// Defines which channel the sound effect will be played through.
    enum SoundEffectChannel:Int {
        /// Play through channel 1.
        case channel01 = 0
        
        /// Play through channel 2.
        case channel02 = 1
    }
    
    // MARK: - Static Properties
    /// Defines the common, shared instance of the Sound Manager
    static var shared:SoundManager = SoundManager()
    
    // MARK: - Properties
    /// Global variable that if `true`, background music will be played in the game.
    @AppStorage("playBackgroundMusic") var shouldPlayBackgroundMusic: Bool = true
    
    /// Global variable that if `true`, sound effect will be played in the game.
    @AppStorage("playSoundEffects") var shouldPlaySoundEffects: Bool = true
    
    /// The `AVAudioPlayer` used to play background music.
    var backgroundMusic:AVAudioPlayer?
    
    /// The sound currently being played in the background music channel.
    var currentBackgroundMusic:String = ""
    
    /// The `AVAudioPlayer` used to play room specific background music.
    var backgroundSound:AVAudioPlayer?
    
    /// The `AVAudioPlayer` used to play the first channel of sound effects.
    var soundEffect01:AVAudioPlayer?
    
    /// The `AVAudioPlayer` used to play  the second channe; of sound effects.
    var soundEffect02:AVAudioPlayer?
    
    /// The delegate used to handle events on the first channel of sound effects.
    private var soundEffectDelegate01:SoundManagerDelegate? = nil
    
    /// The delegate used to handle events on the second channel of sound effects.
    private var soundEffectDelegate02:SoundManagerDelegate? = nil
    
    // MARK: - Functions
    
    /// Starts playing the background music for the given audio file. If the sound is already playing, it will not be restarted. The sound provided will loop forever until `stopBackgroundMusic()` is called.
    /// - Parameter song: The sound (with extension) to be played.
    func startBackgroundMusic(song:String) {
        
        guard shouldPlayBackgroundMusic else {
            return
        }
        
        // Is the song already playing?
        guard currentBackgroundMusic != song else {
            return
        }
        
        if let backgroundMusic = backgroundMusic {
            if backgroundMusic.isPlaying {
                stopBackgroundMusic()
            }
        }
        
        let path = Bundle.main.path(forResource: song, ofType:nil)
        if let path = path {
            let url = URL(fileURLWithPath: path)
            
            do {
                currentBackgroundMusic = song
                backgroundMusic = try AVAudioPlayer(contentsOf: url)
                backgroundMusic?.volume = 0.30
                backgroundMusic?.numberOfLoops = -1
                backgroundMusic?.play()
            } catch {
                print("Unable to play background music: \(error)")
            }
        } else {
            print("Unable find background music: \(song)")
        }
    }
    
    /// Thapls the room specific background music. The sound provided will loop forever until `stopBackgroundSound()` is called.
    /// - Parameter sound: The sound (with extension) to be played.
    func playBackgroundSound(sound:String) {
        
        guard shouldPlayBackgroundMusic else {
            return
        }
        
        if let backgroundSound = backgroundSound {
            if backgroundSound.isPlaying {
                stopBackgroundSound()
            }
        }
        
        let path = Bundle.main.path(forResource: sound, ofType:nil)
        if let path = path {
            let url = URL(fileURLWithPath: path)
            
            do {
                backgroundSound = try AVAudioPlayer(contentsOf: url)
                backgroundSound?.play()
            } catch {
                print("Unable to play background music: \(error)")
            }
        } else {
            print("Unable find background music: \(sound)")
        }
    }
    
    /// Stops the currently playing background music.
    func stopBackgroundMusic() {
        
        backgroundMusic?.stop()
        currentBackgroundMusic = ""
        backgroundMusic = nil
    }
    
    /// Stops the currently playing room specific background music.
    func stopBackgroundSound() {
        
        backgroundSound?.stop()
        backgroundSound = nil
    }
    
    
    /// Plays the given sound effect on the given effect channel.
    /// - Parameters:
    ///   - sound: The sound (with extension) to be played.
    ///   - channel: The effect channel to play the song on. The default is `channel01`.
    ///   - didFinishPlaying: The closure that will be called when the sound finishes playing.
    func playSoundEffect(sound:String, channel:SoundEffectChannel = .channel01, didFinishPlaying:FinishedPlaying? = nil) {
        
        guard shouldPlaySoundEffects else {
            if let didFinishPlaying = didFinishPlaying {
                didFinishPlaying()
            }
            return
        }
        
        let path = Bundle.main.path(forResource: sound, ofType:nil)
        if let path = path {
            let url = URL(fileURLWithPath: path)
            
            do {
                switch(channel) {
                case .channel01:
                    soundEffectDelegate01 = SoundManagerDelegate(action: didFinishPlaying)
                    soundEffect01 = try AVAudioPlayer(contentsOf: url)
                    soundEffect01?.delegate = soundEffectDelegate01
                    soundEffect01?.play()
                case .channel02:
                    soundEffectDelegate02 = SoundManagerDelegate(action: didFinishPlaying)
                    soundEffect02 = try AVAudioPlayer(contentsOf: url)
                    soundEffect02?.delegate = soundEffectDelegate02
                    soundEffect02?.play()
                }
            } catch {
                print("Unable to play sound effect: \(error)")
            }
        } else {
            print("Unable find sound effect: \(sound)")
        }
    }
}

/// Delegate that handles a sound finishing playing on one of the sound effect channels.
class SoundManagerDelegate: NSObject, AVAudioPlayerDelegate {
    typealias FinishedPlaying = () -> Void
    
    // MARK: - Properties
    /// The closure that gets called when the sound finishes playing.
    var finishPlaying:FinishedPlaying? = nil
    
    // MARK: - Initializers
    /// Creates a new instance of the object with the given parameters.
    /// - Parameter action: The closure that gets called when the sound finishes playing.
    init(action:FinishedPlaying?) {
        // Initialize
        self.finishPlaying = action
    }
    
    // MARK: - Functions
    /// Function called when the sound finishes playing
    /// - Parameters:
    ///   - player: The `AVAudioPlayer` that was playing the sound.
    ///   - flag: If `true`, the sound played successfully.
    func audioPlayerDidFinishPlaying(_ player: AVAudioPlayer, successfully flag: Bool) {
        if let finishPlaying = finishPlaying {
            finishPlaying()
        }
    }
}

All playback is handled by a standard AVAudioPlayer. Here’s what all of the functions in the class do:

  • startBackgroundMusic(song:String) – Play the give music on a loop if the user wants to hear background music and the given song is not already playing.
  • playBackgroundSound(sound:String) – Layers a background sound on top of the music that does not loop.
  • stopBackgroundMusic() – Instantly stops any background music.
  • stopBackgroundSound() – Instantly stops any background sound effect.
  • playSoundEffect(sound:String, channel:SoundEffectChannel = .channel01, didFinishPlaying:FinishedPlaying? = nil)– Plays the given sound effect on the given channel. Use didFinishPlaying to take an action after the effect ends.
  • SoundManagerDelegate – Handles a sound effect finishing playing.

When I use this class in the game, I make a call against the shared instance:

import SwiftUI
import SwiftletUtilities
import GameKitUI
import GameKit

struct StartGameView: View {
    @ObservedObject var dataStore = MasterDataStore.sharedDataStore
    ...
    
    var body: some View {
        GeometryReader { geometry in
            ZStack(alignment: .topLeading) {
            
            ...
            
            } 
            .ignoresSafeArea()
        }
        .ignoresSafeArea()
        .onAppear {
            SoundManager.shared.startBackgroundMusic(song: "AnechoixJazzLoop.mp3")
        }
    }
}

Handling Speech Synthesis

To make the game a little quirky and different, I knew I wanted to have each character’s part read aloud. While it would have been nice to have gotten a bunch of voice actors to record all of the different parts, my limited budget just didn’t allow for it.

So I decided to see how far I could push Siri’s text-to-speech functionality. With a little experimentation, I was able to get a couple of different voices programmatically to add a little interest.

Here’s the shared library I ended up creating to handle everything:

import Foundation
import AVFoundation
import SwiftUI

class SpeechManager {
    
    // MARK: - Static Functions
    static var shared:SpeechManager = SpeechManager()
    
    // MARK: - Enumerations
    public enum VoiceLanguage: String {
        case arabicSaudiArabia = "ar-SA"
        case czechCzechRepublic = "cs-CZ"
        case danishDenmark = "da-DK"
        case germanGermany = "de-DE"
        case greekModernGreece = "el-GR"
        case englishAustralia = "en-AU"
        case englishUnitedKingdom = "en-GB"
        case englishIreland = "en-IE"
        case englishIndia = "en-IN"
        case englishUnitedStates = "en-US"
        case englishSouthAfrica = "en-ZA"
        case spanishMexico = "es-MX"
        case spanishSpain = "es-ES"
        case finnishFinland = "fi-FI"
        case frenchCanada = "fr-CA"
        case frenchFrance = "fr-FR"
        case hebrewIsrael = "he-IL"
        case hindiIndia = "hi-IN"
        case indonesianIndonesia = "id-ID"
        case italianItaly = "it-IT"
        case japaneseJapan = "ja-JP"
        case koreanKorea = "ko-KR"
        case dutchBelgium = "nl-BE"
        case dutchNetherlands = "nl-NL"
        case norwegianNorway = "no-NO"
        case polishPoland = "pl-PL"
        case portugueseBrazil = "pt-BR"
        case portuguesePortugal = "pt-PT"
        case romanianRomania = "ro-RO"
        case russianRussianFederation = "ru-RU"
        case slovakSlovakia = "sk-SK"
        case swedishSweden = "sv-SE"
        case thaiThailand = "th-TH"
        case turkishTurkey = "tr-TR"
        case chineseShina = "zh-CN"
        case chineseHongKong = "zh-HK"
        case chineseTaiwan = "zh-TW"
    }
    
    // MARK: - Properties
    @AppStorage("speakText") var speakText: Bool = true
    
    var speechSynthesizer:AVSpeechSynthesizer = AVSpeechSynthesizer()
    
    // MARK: - Functions
    
    /// Uses the default **Speech Synthesizer** to speak the given text aloud.
    /// - Parameter text: The text to read aloud to the user.
    func sayPhrase(_ text:String) {
        guard speakText else {
            return
        }
        
        let speechUtterance = AVSpeechUtterance(string: text)
        speechSynthesizer.speak(speechUtterance)
    }
    
    /// Says the given phrase in the given language.
    /// - Parameters:
    ///   - text: The text to speak.
    ///   - inVoice: The language to speak in.
    func sayPhrase(_ text:String, inVoice:VoiceLanguage) {
        guard speakText else {
            return
        }
        
        let speechSynthesisVoice = AVSpeechSynthesisVoice(language: inVoice.rawValue)
        let speechUtterance = AVSpeechUtterance(string: text)
        speechUtterance.voice = speechSynthesisVoice
        speechSynthesizer.speak(speechUtterance)
    }
    
    /// Stops any speech currently running.
    func stopSpeaking() {
        guard speakText else {
            return
        }
        
        speechSynthesizer.stopSpeaking(at: .word)
    }
}

This code is really pretty simple, the only thing I did “special” was to take all of the languages that can be spoken on iOS and add them to an enum (VoiceLanguage) to make them a little more “human readable”.

In the game I used it like this:

.onAppear{
DispatchQueue.main.async {
    if dataStore.dontAnnounce {
        dataStore.dontAnnounce = false
    } else {
        let person = dataStore.getCharacter(id: dataStore.lastConversation.avatar)!
        if dataStore.lastConversation.sex == .female {
            switch(person.nationality) {
            case .irish:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .englishIreland)
            case .german:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .germanGermany)
            case .british, .french:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .englishSouthAfrica)
            default:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .englishUnitedStates)
            }
        } else {
            switch(person.nationality) {
            case .british, .scottish:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .englishIndia)
            case .french:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .frenchFrance)
            default:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .englishUnitedKingdom)
            }
        }
    }
    
}

The only part that is of real interest here, is that I had to call speech synthesis on the main thread or it caused the app to go crazy.

Support

If you find this useful, please consider making a small donation:

Buy Me A Coffee

It’s through the support of contributors like yourself, I can continue to create useful articles like this one and continue build, release and maintain high-quality, well documented Swift Packages for free.

Closing

For those of you who stuck with me through the entire article, thanks! And I hope you did find some useful information.