Main Article Content
Abstract
Rapid ecological, cultural and social changes pose huge problems for humanity since they represent a great danger to the future of the planet Earth. These current transformations encourage art-science research which has proven to be a valuable means for communication of such challenges. Creative media artists effectively address and critically discuss the impact of cultural and social changes on environment. In the field of virtual heritage, they combine virtual reality and cultural and natural heritage offering an opportunity for the audience to become an integral part of the immersive heritage experiences. This conceptual article offers theoretical reflections on the role of hybrid audiovisual media, as a fusion of virtual reality, music and soundscape, in communicating the importance of natural heritage protection. Conceptual framework of this research combines concepts from new media theory and theory of musical emotions including BRECVEMA framework. Following new media theory, with aim to communicate meaning, this project includes creation of the Virtual Heritage Exhibition, representing natural heritage of Hong Kong, and in particular Hong Kong UNESCO Global Geopark and Indo-Pacific Humpback Dolphin. Musical content and semiotics are in the form of sonification, while visual semiotics are audio-reactive visual effects applied to the natural landmarks in the virtual reality 360 videos. To evaluate if the goal of creating audio-visual virtual heritage environment that allows intercultural communication and learning for the audience is achieved, user experience evaluation is based on the concepts of and theory of musical emotions including BRECVEMA framework.
Keywords
Article Details
Copyright (c) 2024 Mirjana Dokic
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
- Baberini, M., Coleman, C., Slovic, P., & Västfjäll, D. (2015). Examining the effects of photographic attributes on sympathy, emotions, and donation behavior. Visual Communication Quarterly, 22(2), 118–128. Baberini, M., Coleman, C., Slovic, P., & Västfjäll, D. (2015). Examining theeffects of photographic attributes on sympathy, emotions, and donation behavior. Visual Communication Quarterly, 22(2), 118–128. https://doi.org/10.1080/15551393.2015.1061433
- Barnosky, A.D., Matzke, N., Tomiya, S., Wogan, G.O.U., Swartz, B., Quental, T.B., Marshall, C., McGuire, J.L., Lindsey, E.L., Maguire, K.C., Mersey, B., Ferrer, E.A., (2011). Has the Earth’s sixth mass extinction already arrived? Nature 471, 51–57.
- Baumgartner, H. (1992). Remembrance of things past: Music, autobiographical memory, and emotion. Advances in Consumer Research , 19, 613–620.
- Blair, M. E., & Shimp, T. A. (1992). Consequences of an unpleasant experience with music: A second-order negative conditioning perspective. Journal of Advertising, 21, 35–43. https://doi.org/10.1080/00913367.1992.10673358
- Brondizio, E., Settele, J., Díaz, S., Ngo, H. (2019). Global assessment report on biodiversity and ecosystem services of the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services. IPBES Secretariat, 2019.
- Champion, E. M. (2021). Virtual Heritage: A Guide. London: Ubiquity Press.DOI: https://doi.org/10.5334/bck.
- Dowling W. J., & Harwood, D. L. (1986). Music cognition. New York: Academic Press.
- Grassini, S., Ratcliffe, E. (2024). The Virtual Wild: Exploring the Intersection of Virtual Reality and Natural Environments. In: Finneran, N., Hewlett, D., Clarke, R. (eds) Managing Protected Areas. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-031-40783-3_17
- Gutheil, E. A. (1952). Introduction. In A. Carpurso et al. (Eds.), Music and your emotions: A practical guide to music selections associated with desired emotional responses (pp. 9- 13). New York: Liveright.
- Harrer, G., & Harrer, H. (1977). Music, emotion, and autonomic function. In M. Critchley & R. A. Henson (Eds.), Music and the brain: Studies in the neurology of music (pp. 202–216). London, England: William Heineman
- Holmes, D. (2012). "New Media Theory." Encyclopedia of Communication Theory. Ed. . Thousand Oaks, CA: SAGE, 2009. 685-89. SAGE Reference Online. Web. 29 Jun. 2012.
- Juslin, P. N. (2000). Vocal expression and musical expression: Parallels and contrasts. In A. Kappas (Ed.), Proceedings of the Sixteenth Conference of the International Society for Research on Emotions (pp. 281–284). ISRE Publications.
- Juslin, P. N. (2001). Communicating emotion in music performance: A review and a theoretical framework. In P. N. Juslin & J. A. Sloboda (Eds.), Music and emotion: Theory and research (pp. 309-337). Oxford: Oxford University Press.
- Juslin, P. N., & Laukka, P. (2004). Expression, perception, and induction of musical emotions: A review and a questionnaire study of everyday listening. Journal of New Music Research, 33, 217-238.
- Juslin, P. N., & Västfjäll, D. (2008). Emotional responses to music: The need to consider underlying mechanisms. Behavioral and Brain Sciences, 31(5), 559–575. doi:10.1017/S0140525X08005293
- Juslin, P. N., Liljeström, S., Västfjäll, D., Barradas, G., & Silva, A. (2008). An experience sampling study of emotional reactions to music: Listener, music, and situation. Emotion, 8(5), 668–683. https://doi.org/10.1037/a0013505
- Juslin, P. N., Liljeström, S., Västfjäll, D., & Lundqvist, L.-O. (2010). How does music evoke emotions? Exploring the underlying mechanisms. In P. N. Juslin & J. A. Sloboda (Eds.), Handbook of music and emotion: Theory, research, applications (pp. 605-642). Oxford: Oxford University Press.
- Juslin, P. (2011). Music and emotion: seven questions, seven answers. In Oxford University Press eBooks (pp. 113–136). https://doi.org/10.1093/acprof:osobl/9780199581566.003.0007
- Juslin P. N. (2013). From everyday emotions to aesthetic emotions: towards a unified theory of musical emotions. Physics of life reviews , 10(3), 235–266. https://doi.org/10.1016/j.plrev.2013.05.008
- Juslin, P., Barradas, G., & Eerola, T. (2015). From Sound to Significance: Exploring the mechanisms underlying emotional reactions to music. American Journal of Psychology , 128(3), 281–304. https://doi.org/10.5406/amerjpsyc.128.3.0281
- Kagan, S. and Kirchberg, V. (2016). Music and sustainability: organizational cultures towards creative resilience – a review. Journal of Cleaner Production 135:1487-1502. http://dx.doi. org/10.1016/j.jclepro.2016.05.044
- Littlejohn, S. W. and Floss. K.A. (2009). Encyclopedia of Communication Theory. USA: SAGE.684
- McLuhan, Marshall (1964). Understanding Media: The Extensions of Man. McGraw-Hill pp. 8, 9, 12, 13. ISBN 81-14-67535-7
- Meyer, L. B. (1956). Emotion and meaning in music. Chicago, IL: Chicago University Press.
- Moser, S. C., & Dilling, L. (2011). COMMUNICATING CHANGE SCIENCE:-CLOSING ACTION CLIMATE. The Oxford handbook of climate change and society, 161. Millar, M., Westfall, R. S., & Fink-Armold
- Nelson, K., Schlüter, A., & Vance, C. (2018a). Distributional preferences and donation behavior among marine resource users in Wakatobi, Indonesia.Ocean and Coastal Management, 162, 34–45. https://doi.org/10.1016/j.ocecoaman.2017.09.003
- Ortony, A., Clore, G., & Collins, A. (1988). The cognitive structure of emotions. Cambridge: Cambridge University Press.
- Patel, A.D. (2008). Music, language, and brain. New York: Oxford University Press
- Pimm, S.L., Jenkins, C.N., Abell, R., Brooks, T.M., Gittleman, J.L., Joppa, L.N., Raven, P. H., Roberts, C.M., Sexton, J.O. (2014). The biodiversity of species and their rates of extinction, distribution, and protection. Science 344, 1246752.
- Platform on Biodiversity and Ecosystem Services. (2021). IPBES Secretariat Intergovernmental Panel on Climate Change (IPCC). Summary for Policymakers. In: Masson-Delmotte, V., Zhai, P., Pirani, A., Connors, S.L., Pean, C., Berger, S., Caud, N., Chen, Y., Goldfarb, L., Gomis, M.I., Huang, M., Leitzell, K., Lonnoy, E., Matthews, J.B.R., Maycock, T.K., Waterfield, T., Yelekçi, O., Yu, R., Zhou, B. (Eds.) Climate
- Privitera, A.G., Fontana, F. and Geronazzo, M. (2024). The Role of Audio in Immersive Storytelling: a Systematic Review in Cultural Heritage. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-024-19288-4
- Poster, M. (1995). The second media age. In EBL. Polity Press ; B. Blackwell. https://archive.org/details/secondmediaage0000post
- Reaver, K. (2019). Three case studies in virtual preservation. Applying virtual reality to Cultural Heritage. AGATHÓN – International Journal of Architecture, Art and Design. Vol. 6, no. online. pp. 210-217 https://doi.org/10.19229/2464-9309/6202019
- Scheffer, M., J. Bascompte, T. K. Bjordam, S. R. Carpenter, L. B. Clarke, C. Folke, P. Marquet, N. Mazzeo, M. Meerhoff, O. Sala, and F. R. Westley. (2015). Dual thinking for scientists. Ecology and Society 20(2):3. http://dx.doi.org/10.5751/ES-07434-200203
- Scherer, K. R., & Zentner, M. R. (2001). Emotional effects of music: Production rules. In P. N. Juslin & J. A. Sloboda (Eds.), Music and emotion: Theory and research (pp. 361- 392). Oxford: Oxford University Press.
- Shin, D. (2018). Empathy and embodied experience in virtual environment: To what extent can virtual reality stimulate empathy and embodied experience? Computers in Human Behavior, 78, 64–73. https://doi.org/10.1016/j.chb.2017.09.012
- Sloboda, J. A. (1996). Emotional responses to music: A review. In K. Riederer, & T. Lahti (Eds.), Proceedings of the Nordic Acoustical Meeting (pp. 385-392). Helsinki: The Acoustical Society of Finland.
- Terra Mosana (2018) Home. Available at: http://www.terramosana.org/ (Accessed: 27 June 2019).
- Thompson, W. F. (2009). Music, thought, and feeling. Understanding the psychology of music. Oxford: Oxford University Press.
- Voosen, P. (2020). Seas are rising faster than ever. Science 370, 901.
- Waterman, M. (1996). Emotional responses to music: Implicit and explicit effects in listeners and performers. Psychology of Music, 24, 53-67.
- World Meteorological Organization (WMO) (2021).The State of the Global Climate 2021. https://public.wmo.int/en/our-mandate/climate/wmo-statement-state-of-global-climate
References
Baberini, M., Coleman, C., Slovic, P., & Västfjäll, D. (2015). Examining the effects of photographic attributes on sympathy, emotions, and donation behavior. Visual Communication Quarterly, 22(2), 118–128. Baberini, M., Coleman, C., Slovic, P., & Västfjäll, D. (2015). Examining theeffects of photographic attributes on sympathy, emotions, and donation behavior. Visual Communication Quarterly, 22(2), 118–128. https://doi.org/10.1080/15551393.2015.1061433
Barnosky, A.D., Matzke, N., Tomiya, S., Wogan, G.O.U., Swartz, B., Quental, T.B., Marshall, C., McGuire, J.L., Lindsey, E.L., Maguire, K.C., Mersey, B., Ferrer, E.A., (2011). Has the Earth’s sixth mass extinction already arrived? Nature 471, 51–57.
Baumgartner, H. (1992). Remembrance of things past: Music, autobiographical memory, and emotion. Advances in Consumer Research , 19, 613–620.
Blair, M. E., & Shimp, T. A. (1992). Consequences of an unpleasant experience with music: A second-order negative conditioning perspective. Journal of Advertising, 21, 35–43. https://doi.org/10.1080/00913367.1992.10673358
Brondizio, E., Settele, J., Díaz, S., Ngo, H. (2019). Global assessment report on biodiversity and ecosystem services of the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services. IPBES Secretariat, 2019.
Champion, E. M. (2021). Virtual Heritage: A Guide. London: Ubiquity Press.DOI: https://doi.org/10.5334/bck.
Dowling W. J., & Harwood, D. L. (1986). Music cognition. New York: Academic Press.
Grassini, S., Ratcliffe, E. (2024). The Virtual Wild: Exploring the Intersection of Virtual Reality and Natural Environments. In: Finneran, N., Hewlett, D., Clarke, R. (eds) Managing Protected Areas. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-031-40783-3_17
Gutheil, E. A. (1952). Introduction. In A. Carpurso et al. (Eds.), Music and your emotions: A practical guide to music selections associated with desired emotional responses (pp. 9- 13). New York: Liveright.
Harrer, G., & Harrer, H. (1977). Music, emotion, and autonomic function. In M. Critchley & R. A. Henson (Eds.), Music and the brain: Studies in the neurology of music (pp. 202–216). London, England: William Heineman
Holmes, D. (2012). "New Media Theory." Encyclopedia of Communication Theory. Ed. . Thousand Oaks, CA: SAGE, 2009. 685-89. SAGE Reference Online. Web. 29 Jun. 2012.
Juslin, P. N. (2000). Vocal expression and musical expression: Parallels and contrasts. In A. Kappas (Ed.), Proceedings of the Sixteenth Conference of the International Society for Research on Emotions (pp. 281–284). ISRE Publications.
Juslin, P. N. (2001). Communicating emotion in music performance: A review and a theoretical framework. In P. N. Juslin & J. A. Sloboda (Eds.), Music and emotion: Theory and research (pp. 309-337). Oxford: Oxford University Press.
Juslin, P. N., & Laukka, P. (2004). Expression, perception, and induction of musical emotions: A review and a questionnaire study of everyday listening. Journal of New Music Research, 33, 217-238.
Juslin, P. N., & Västfjäll, D. (2008). Emotional responses to music: The need to consider underlying mechanisms. Behavioral and Brain Sciences, 31(5), 559–575. doi:10.1017/S0140525X08005293
Juslin, P. N., Liljeström, S., Västfjäll, D., Barradas, G., & Silva, A. (2008). An experience sampling study of emotional reactions to music: Listener, music, and situation. Emotion, 8(5), 668–683. https://doi.org/10.1037/a0013505
Juslin, P. N., Liljeström, S., Västfjäll, D., & Lundqvist, L.-O. (2010). How does music evoke emotions? Exploring the underlying mechanisms. In P. N. Juslin & J. A. Sloboda (Eds.), Handbook of music and emotion: Theory, research, applications (pp. 605-642). Oxford: Oxford University Press.
Juslin, P. (2011). Music and emotion: seven questions, seven answers. In Oxford University Press eBooks (pp. 113–136). https://doi.org/10.1093/acprof:osobl/9780199581566.003.0007
Juslin P. N. (2013). From everyday emotions to aesthetic emotions: towards a unified theory of musical emotions. Physics of life reviews , 10(3), 235–266. https://doi.org/10.1016/j.plrev.2013.05.008
Juslin, P., Barradas, G., & Eerola, T. (2015). From Sound to Significance: Exploring the mechanisms underlying emotional reactions to music. American Journal of Psychology , 128(3), 281–304. https://doi.org/10.5406/amerjpsyc.128.3.0281
Kagan, S. and Kirchberg, V. (2016). Music and sustainability: organizational cultures towards creative resilience – a review. Journal of Cleaner Production 135:1487-1502. http://dx.doi. org/10.1016/j.jclepro.2016.05.044
Littlejohn, S. W. and Floss. K.A. (2009). Encyclopedia of Communication Theory. USA: SAGE.684
McLuhan, Marshall (1964). Understanding Media: The Extensions of Man. McGraw-Hill pp. 8, 9, 12, 13. ISBN 81-14-67535-7
Meyer, L. B. (1956). Emotion and meaning in music. Chicago, IL: Chicago University Press.
Moser, S. C., & Dilling, L. (2011). COMMUNICATING CHANGE SCIENCE:-CLOSING ACTION CLIMATE. The Oxford handbook of climate change and society, 161. Millar, M., Westfall, R. S., & Fink-Armold
Nelson, K., Schlüter, A., & Vance, C. (2018a). Distributional preferences and donation behavior among marine resource users in Wakatobi, Indonesia.Ocean and Coastal Management, 162, 34–45. https://doi.org/10.1016/j.ocecoaman.2017.09.003
Ortony, A., Clore, G., & Collins, A. (1988). The cognitive structure of emotions. Cambridge: Cambridge University Press.
Patel, A.D. (2008). Music, language, and brain. New York: Oxford University Press
Pimm, S.L., Jenkins, C.N., Abell, R., Brooks, T.M., Gittleman, J.L., Joppa, L.N., Raven, P. H., Roberts, C.M., Sexton, J.O. (2014). The biodiversity of species and their rates of extinction, distribution, and protection. Science 344, 1246752.
Platform on Biodiversity and Ecosystem Services. (2021). IPBES Secretariat Intergovernmental Panel on Climate Change (IPCC). Summary for Policymakers. In: Masson-Delmotte, V., Zhai, P., Pirani, A., Connors, S.L., Pean, C., Berger, S., Caud, N., Chen, Y., Goldfarb, L., Gomis, M.I., Huang, M., Leitzell, K., Lonnoy, E., Matthews, J.B.R., Maycock, T.K., Waterfield, T., Yelekçi, O., Yu, R., Zhou, B. (Eds.) Climate
Privitera, A.G., Fontana, F. and Geronazzo, M. (2024). The Role of Audio in Immersive Storytelling: a Systematic Review in Cultural Heritage. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-024-19288-4
Poster, M. (1995). The second media age. In EBL. Polity Press ; B. Blackwell. https://archive.org/details/secondmediaage0000post
Reaver, K. (2019). Three case studies in virtual preservation. Applying virtual reality to Cultural Heritage. AGATHÓN – International Journal of Architecture, Art and Design. Vol. 6, no. online. pp. 210-217 https://doi.org/10.19229/2464-9309/6202019
Scheffer, M., J. Bascompte, T. K. Bjordam, S. R. Carpenter, L. B. Clarke, C. Folke, P. Marquet, N. Mazzeo, M. Meerhoff, O. Sala, and F. R. Westley. (2015). Dual thinking for scientists. Ecology and Society 20(2):3. http://dx.doi.org/10.5751/ES-07434-200203
Scherer, K. R., & Zentner, M. R. (2001). Emotional effects of music: Production rules. In P. N. Juslin & J. A. Sloboda (Eds.), Music and emotion: Theory and research (pp. 361- 392). Oxford: Oxford University Press.
Shin, D. (2018). Empathy and embodied experience in virtual environment: To what extent can virtual reality stimulate empathy and embodied experience? Computers in Human Behavior, 78, 64–73. https://doi.org/10.1016/j.chb.2017.09.012
Sloboda, J. A. (1996). Emotional responses to music: A review. In K. Riederer, & T. Lahti (Eds.), Proceedings of the Nordic Acoustical Meeting (pp. 385-392). Helsinki: The Acoustical Society of Finland.
Terra Mosana (2018) Home. Available at: http://www.terramosana.org/ (Accessed: 27 June 2019).
Thompson, W. F. (2009). Music, thought, and feeling. Understanding the psychology of music. Oxford: Oxford University Press.
Voosen, P. (2020). Seas are rising faster than ever. Science 370, 901.
Waterman, M. (1996). Emotional responses to music: Implicit and explicit effects in listeners and performers. Psychology of Music, 24, 53-67.
World Meteorological Organization (WMO) (2021).The State of the Global Climate 2021. https://public.wmo.int/en/our-mandate/climate/wmo-statement-state-of-global-climate