Vivid Background Audio Generation based on Large Language Models and AudioLDM

This paper describes a background audio and speech fusion generation system. Our main arguments include background audio generation based on latent diffusion, text-to-speech (TTS) based on speech synthesis, and text information extraction based on large language models. We compared the influence of text information extraction on the degree of correlation between background audio and its corresponding speech. At the same time, the results of different large language models on the audio generated after information extraction are compared. We also propose a new evaluation metric named Overall Correlation Quality Score (OCQS) to evaluate the naturalness and rationality between speech text and its background audio. With the above evaluation metric, We tested multiple models and eventually found that the background audio generated by speech text extracted and summarized by a large language model had better quality.