Adventures in SwiftUI & Game Dev – Bonus Round

by | Oct 4, 2021 | Blog, Board Games, Games

In final part of my multipart article on converting my Murdered by Midnight Board Game to a Mobile Game App using Swift and SwiftUI, I’ll cover supporting Music, Sound Effects & Voice in my Multiplatform SwiftUI game.

Hopefully, other indie developers can find this information useful.

Index:

  • Part 1 – Deals with converting a mockup to SwiftUI, handling different device sizes & screen ratios and displays screens as needed.
  • Part 2 – Covers adding Game Center support to a SwiftUI app.
  • Part 3 – Covers handling tvOS quirks in a SwiftUI app.
  • Part 4 – Covers handling macOS Catalyst quirks in a SwiftUI app.
  • Bonus Round – Covers handling Music, Sound Effects & Voice in a SwiftUI app.

Handling Background Music and Sound Effects

I knew I want to have background music in my game, along with the ability to play a few sound effects at the same time, over this music. Here is the common class that I cooked up to handle everything:

import Foundation
import AVFoundation
import SwiftUI

/// Class to handle playing background music and sound effects throughout the game.
class SoundManager: NSObject, AVAudioPlayerDelegate {
    typealias FinishedPlaying = () -> Void
    
    // MARK: - Enumerations
    /// Defines which channel the sound effect will be played through.
    enum SoundEffectChannel:Int {
        /// Play through channel 1.
        case channel01 = 0
        
        /// Play through channel 2.
        case channel02 = 1
    }
    
    // MARK: - Static Properties
    /// Defines the common, shared instance of the Sound Manager
    static var shared:SoundManager = SoundManager()
    
    // MARK: - Properties
    /// Global variable that if `true`, background music will be played in the game.
    @AppStorage("playBackgroundMusic") var shouldPlayBackgroundMusic: Bool = true
    
    /// Global variable that if `true`, sound effect will be played in the game.
    @AppStorage("playSoundEffects") var shouldPlaySoundEffects: Bool = true
    
    /// The `AVAudioPlayer` used to play background music.
    var backgroundMusic:AVAudioPlayer?
    
    /// The sound currently being played in the background music channel.
    var currentBackgroundMusic:String = ""
    
    /// The `AVAudioPlayer` used to play room specific background music.
    var backgroundSound:AVAudioPlayer?
    
    /// The `AVAudioPlayer` used to play the first channel of sound effects.
    var soundEffect01:AVAudioPlayer?
    
    /// The `AVAudioPlayer` used to play  the second channe; of sound effects.
    var soundEffect02:AVAudioPlayer?
    
    /// The delegate used to handle events on the first channel of sound effects.
    private var soundEffectDelegate01:SoundManagerDelegate? = nil
    
    /// The delegate used to handle events on the second channel of sound effects.
    private var soundEffectDelegate02:SoundManagerDelegate? = nil
    
    // MARK: - Functions
    
    /// Starts playing the background music for the given audio file. If the sound is already playing, it will not be restarted. The sound provided will loop forever until `stopBackgroundMusic()` is called.
    /// - Parameter song: The sound (with extension) to be played.
    func startBackgroundMusic(song:String) {
        
        guard shouldPlayBackgroundMusic else {
            return
        }
        
        // Is the song already playing?
        guard currentBackgroundMusic != song else {
            return
        }
        
        if let backgroundMusic = backgroundMusic {
            if backgroundMusic.isPlaying {
                stopBackgroundMusic()
            }
        }
        
        let path = Bundle.main.path(forResource: song, ofType:nil)
        if let path = path {
            let url = URL(fileURLWithPath: path)
            
            do {
                currentBackgroundMusic = song
                backgroundMusic = try AVAudioPlayer(contentsOf: url)
                backgroundMusic?.volume = 0.30
                backgroundMusic?.numberOfLoops = -1
                backgroundMusic?.play()
            } catch {
                print("Unable to play background music: \(error)")
            }
        } else {
            print("Unable find background music: \(song)")
        }
    }
    
    /// Thapls the room specific background music. The sound provided will loop forever until `stopBackgroundSound()` is called.
    /// - Parameter sound: The sound (with extension) to be played.
    func playBackgroundSound(sound:String) {
        
        guard shouldPlayBackgroundMusic else {
            return
        }
        
        if let backgroundSound = backgroundSound {
            if backgroundSound.isPlaying {
                stopBackgroundSound()
            }
        }
        
        let path = Bundle.main.path(forResource: sound, ofType:nil)
        if let path = path {
            let url = URL(fileURLWithPath: path)
            
            do {
                backgroundSound = try AVAudioPlayer(contentsOf: url)
                backgroundSound?.play()
            } catch {
                print("Unable to play background music: \(error)")
            }
        } else {
            print("Unable find background music: \(sound)")
        }
    }
    
    /// Stops the currently playing background music.
    func stopBackgroundMusic() {
        
        backgroundMusic?.stop()
        currentBackgroundMusic = ""
        backgroundMusic = nil
    }
    
    /// Stops the currently playing room specific background music.
    func stopBackgroundSound() {
        
        backgroundSound?.stop()
        backgroundSound = nil
    }
    
    
    /// Plays the given sound effect on the given effect channel.
    /// - Parameters:
    ///   - sound: The sound (with extension) to be played.
    ///   - channel: The effect channel to play the song on. The default is `channel01`.
    ///   - didFinishPlaying: The closure that will be called when the sound finishes playing.
    func playSoundEffect(sound:String, channel:SoundEffectChannel = .channel01, didFinishPlaying:FinishedPlaying? = nil) {
        
        guard shouldPlaySoundEffects else {
            if let didFinishPlaying = didFinishPlaying {
                didFinishPlaying()
            }
            return
        }
        
        let path = Bundle.main.path(forResource: sound, ofType:nil)
        if let path = path {
            let url = URL(fileURLWithPath: path)
            
            do {
                switch(channel) {
                case .channel01:
                    soundEffectDelegate01 = SoundManagerDelegate(action: didFinishPlaying)
                    soundEffect01 = try AVAudioPlayer(contentsOf: url)
                    soundEffect01?.delegate = soundEffectDelegate01
                    soundEffect01?.play()
                case .channel02:
                    soundEffectDelegate02 = SoundManagerDelegate(action: didFinishPlaying)
                    soundEffect02 = try AVAudioPlayer(contentsOf: url)
                    soundEffect02?.delegate = soundEffectDelegate02
                    soundEffect02?.play()
                }
            } catch {
                print("Unable to play sound effect: \(error)")
            }
        } else {
            print("Unable find sound effect: \(sound)")
        }
    }
}

/// Delegate that handles a sound finishing playing on one of the sound effect channels.
class SoundManagerDelegate: NSObject, AVAudioPlayerDelegate {
    typealias FinishedPlaying = () -> Void
    
    // MARK: - Properties
    /// The closure that gets called when the sound finishes playing.
    var finishPlaying:FinishedPlaying? = nil
    
    // MARK: - Initializers
    /// Creates a new instance of the object with the given parameters.
    /// - Parameter action: The closure that gets called when the sound finishes playing.
    init(action:FinishedPlaying?) {
        // Initialize
        self.finishPlaying = action
    }
    
    // MARK: - Functions
    /// Function called when the sound finishes playing
    /// - Parameters:
    ///   - player: The `AVAudioPlayer` that was playing the sound.
    ///   - flag: If `true`, the sound played successfully.
    func audioPlayerDidFinishPlaying(_ player: AVAudioPlayer, successfully flag: Bool) {
        if let finishPlaying = finishPlaying {
            finishPlaying()
        }
    }
}

All playback is handled by a standard AVAudioPlayer. Here’s what all of the functions in the class do:

  • startBackgroundMusic(song:String) – Play the give music on a loop if the user wants to hear background music and the given song is not already playing.
  • playBackgroundSound(sound:String) – Layers a background sound on top of the music that does not loop.
  • stopBackgroundMusic() – Instantly stops any background music.
  • stopBackgroundSound() – Instantly stops any background sound effect.
  • playSoundEffect(sound:String, channel:SoundEffectChannel = .channel01, didFinishPlaying:FinishedPlaying? = nil)– Plays the given sound effect on the given channel. Use didFinishPlaying to take an action after the effect ends.
  • SoundManagerDelegate – Handles a sound effect finishing playing.

When I use this class in the game, I make a call against the shared instance:

import SwiftUI
import SwiftletUtilities
import GameKitUI
import GameKit

struct StartGameView: View {
    @ObservedObject var dataStore = MasterDataStore.sharedDataStore
    ...
    
    var body: some View {
        GeometryReader { geometry in
            ZStack(alignment: .topLeading) {
            
            ...
            
            } 
            .ignoresSafeArea()
        }
        .ignoresSafeArea()
        .onAppear {
            SoundManager.shared.startBackgroundMusic(song: "AnechoixJazzLoop.mp3")
        }
    }
}

Handling Speech Synthesis

To make the game a little quirky and different, I knew I wanted to have each character’s part read aloud. While it would have been nice to have gotten a bunch of voice actors to record all of the different parts, my limited budget just didn’t allow for it.

So I decided to see how far I could push Siri’s text-to-speech functionality. With a little experimentation, I was able to get a couple of different voices programmatically to add a little interest.

Here’s the shared library I ended up creating to handle everything:

import Foundation
import AVFoundation
import SwiftUI

class SpeechManager {
    
    // MARK: - Static Functions
    static var shared:SpeechManager = SpeechManager()
    
    // MARK: - Enumerations
    public enum VoiceLanguage: String {
        case arabicSaudiArabia = "ar-SA"
        case czechCzechRepublic = "cs-CZ"
        case danishDenmark = "da-DK"
        case germanGermany = "de-DE"
        case greekModernGreece = "el-GR"
        case englishAustralia = "en-AU"
        case englishUnitedKingdom = "en-GB"
        case englishIreland = "en-IE"
        case englishIndia = "en-IN"
        case englishUnitedStates = "en-US"
        case englishSouthAfrica = "en-ZA"
        case spanishMexico = "es-MX"
        case spanishSpain = "es-ES"
        case finnishFinland = "fi-FI"
        case frenchCanada = "fr-CA"
        case frenchFrance = "fr-FR"
        case hebrewIsrael = "he-IL"
        case hindiIndia = "hi-IN"
        case indonesianIndonesia = "id-ID"
        case italianItaly = "it-IT"
        case japaneseJapan = "ja-JP"
        case koreanKorea = "ko-KR"
        case dutchBelgium = "nl-BE"
        case dutchNetherlands = "nl-NL"
        case norwegianNorway = "no-NO"
        case polishPoland = "pl-PL"
        case portugueseBrazil = "pt-BR"
        case portuguesePortugal = "pt-PT"
        case romanianRomania = "ro-RO"
        case russianRussianFederation = "ru-RU"
        case slovakSlovakia = "sk-SK"
        case swedishSweden = "sv-SE"
        case thaiThailand = "th-TH"
        case turkishTurkey = "tr-TR"
        case chineseShina = "zh-CN"
        case chineseHongKong = "zh-HK"
        case chineseTaiwan = "zh-TW"
    }
    
    // MARK: - Properties
    @AppStorage("speakText") var speakText: Bool = true
    
    var speechSynthesizer:AVSpeechSynthesizer = AVSpeechSynthesizer()
    
    // MARK: - Functions
    
    /// Uses the default **Speech Synthesizer** to speak the given text aloud.
    /// - Parameter text: The text to read aloud to the user.
    func sayPhrase(_ text:String) {
        guard speakText else {
            return
        }
        
        let speechUtterance = AVSpeechUtterance(string: text)
        speechSynthesizer.speak(speechUtterance)
    }
    
    /// Says the given phrase in the given language.
    /// - Parameters:
    ///   - text: The text to speak.
    ///   - inVoice: The language to speak in.
    func sayPhrase(_ text:String, inVoice:VoiceLanguage) {
        guard speakText else {
            return
        }
        
        let speechSynthesisVoice = AVSpeechSynthesisVoice(language: inVoice.rawValue)
        let speechUtterance = AVSpeechUtterance(string: text)
        speechUtterance.voice = speechSynthesisVoice
        speechSynthesizer.speak(speechUtterance)
    }
    
    /// Stops any speech currently running.
    func stopSpeaking() {
        guard speakText else {
            return
        }
        
        speechSynthesizer.stopSpeaking(at: .word)
    }
}

This code is really pretty simple, the only thing I did “special” was to take all of the languages that can be spoken on iOS and add them to an enum (VoiceLanguage) to make them a little more “human readable”.

In the game I used it like this:

.onAppear{
DispatchQueue.main.async {
    if dataStore.dontAnnounce {
        dataStore.dontAnnounce = false
    } else {
        let person = dataStore.getCharacter(id: dataStore.lastConversation.avatar)!
        if dataStore.lastConversation.sex == .female {
            switch(person.nationality) {
            case .irish:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .englishIreland)
            case .german:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .germanGermany)
            case .british, .french:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .englishSouthAfrica)
            default:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .englishUnitedStates)
            }
        } else {
            switch(person.nationality) {
            case .british, .scottish:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .englishIndia)
            case .french:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .frenchFrance)
            default:
                SpeechManager.shared.sayPhrase(dataStore.lastConversation.text, inVoice: .englishUnitedKingdom)
            }
        }
    }
    
}

The only part that is of real interest here, is that I had to call speech synthesis on the main thread or it caused the app to go crazy.

Closing

For those of you who stuck with me through the entire article, thanks! And I hope you did find some useful information.