Transferring timbre from one sound to another; Synthesis by example. There are a lot of ways this could be done, but the classic is the “talking orchestra” vocoder, which always seemed ham fisted to me. I think about it more in terms of sprase_coding, but there are many ways to think about it. When you refer to “concatenative synthesis” or an “Audio mosaic”, you usually mean using a granular synthesis method. This being the epoch of neural networks, someone will probably get style transfer for audio functioning soon.
The most comprehensive overview of classic concatenativge style stuff IMO is contained in Graham Coleman’s doctoral dissertation, below, which frames it in terms of loss functions and descriptors. (Cole15)
There are a few classic implementations about;
Related: analysis-resynthesis, learning gamelan.
- GaEB15: (2015) A Neural Algorithm of Artistic Style. ArXiv:1508.06576 [Cs, q-Bio].
- MaBC97a: (1997a) A review of time–frequency representations, with application to sound/music analysis–resynthesis. Organised Sound, 2(03), 193–205. DOI
- Stur06: (2006) Adaptive concatenative sound synthesis and its application to micromontage composition. Computer Music Journal, 30(4), 46–66. DOI
- SRMS09: (2009) Analysis, visualization, and transformation of audio signals using dictionary-based methods. Journal of New Music Research, 38(4), 325–341. DOI
- SBSA05: (2005) Audio analogies: Creating new music from an existing performance by concatenative synthesis. In Proceedings of the 2005 International Computer Music Conference (pp. 65–72).
- StCh12: (2012) Comparison of orthogonal matching pursuit implementations. In 20th European Signal Processing Conference (EUSIPCO) (pp. 220–224).
- Stur01: (2001) Composing for an ensemble of atoms: The metamorphosis of scientific experiment into music. Organised Sound, 6, 131–145. DOI
- ABLA03: (2003) Content-based Transformations. Journal of New Music Research, 32(1), 95–114. DOI
- Schw07: (2007) Corpus-based concatenative synthesis. IEEE Signal Processing Magazine, 24(2), 92–104. DOI
- MiGS13: (2013) Corpus-based visual synthesis: an approach for artistic stylization. (p. 51). ACM Press DOI
- ElCM08: (2008) Cross-correlation of beat-synchronous representations for music similarity. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2008. ICASSP 2008 (pp. 57–60). DOI
- SSDR08: (2008) Dark Energy in Sparse Atomic Estimations. Trans. Audio, Speech and Lang. Proc., 16(3), 671–676. DOI
- Coll12: (2012) Even More Errant Sound Synthesis.
- CoDA07: (2007) GUIDAGE: A fast audio query guided assemblage.. ICMA
- DrME14: (2014) Improving time-scale modification of music signals using harmonic-percussive separation. IEEE Signal Processing Letters, 21(1), 105–109. DOI
- Neid10: (2010) Introduction to Automatic Differentiation and MATLAB Object-Oriented Programming. SIAM Review, 52(3), 545–563. DOI
- AuPa06: (2006) Jamming with Plunderphonics: Interactive concatenative synthesis of music. Journal of New Music Research, 35(1), 35–50. DOI
- KrGY97: (1997) Modelling of natural sounds by time–frequency and wavelet representations. Organised Sound, 2(03), 179–191. DOI
- BlDa04: (2004) On Shift-Invariant Sparse Coding. In Independent Component Analysis and Blind Signal Separation (Vol. 3195, pp. 1205–1212). Berlin, Heidelberg: Springer Berlin Heidelberg
- ReLo02: (2002) Optimized orthogonal matching pursuit approach. IEEE Signal Processing Letters, 9(4), 137–140. DOI
- FoKe09: (2009) Recent advances in surrogate-based optimization. Progress in Aerospace Sciences, 45(1–3), 50–79. DOI
- GrBa84: (1984) Representing periodic waveforms with nonorthogonal basis functions. IEEE Transactions on Circuits and Systems, 31(6), 518–534. DOI
- TaOS14: (2014) Singing voice enhancement in monaural music signals based on two-stage harmonic/percussive sound separation on multiple resolution spectrograms. Audio, Speech, and Language Processing, IEEE/ACM Transactions On, 22(1), 228–237. DOI
- KoSD13: (2013) Social Sparsity! Neighborhood Systems Enrich Structured Shrinkage Operators. IEEE Transactions on Signal Processing, 61(10), 2498–2511. DOI
- CoSt00: (n.d.) Sound cross-synthesis and morphing using dictionary-based methods. In International Computer Music Conference.
- KePu10: (2010) Sound texture synthesis with hidden markov tree models in the wavelet domain.
- BlDa06: (2006) Sparse and shift-Invariant representations of music. IEEE Transactions on Audio, Speech and Language Processing, 14(1), 50–57. DOI
- Stur09: (2009) Sparse approximation and atomic decomposition: considering atom interactions in evaluating and building signal representations (phdthesis). University of California, Santa Barbara, CA
- Stur11: (2011) Sparse vector distributions and recovery from compressed sensing.
- Schw11: (2011) State of the art in sound texture synthesis. In Proceedings of DAFx-11 (pp. 221–231).
- QHSG05: (2005) Surrogate-based analysis and optimization. Progress in Aerospace Sciences, 41(1), 1–28. DOI
- MaBC97b: (1997b) The importance of the time–frequency representation for sound/music analysis–resynthesis. Organised Sound, 2(03), 207–214. DOI