Sonification of a NSW storm (8/12/2010)
Rainfall sonification recording of a NSW storm (8/12/2010) : A brief research report for the Rainwire project
***NOV 2011 UPDATE***
Two Rainwire research papers have been accepted for publication:
Burraston, D. (2012) ‘Rainwire: Environmental Soniﬁcation of Rainfall’, Leonardo, MIT Press, (forthcoming) pre-publication PDF @ Leonardo website
Burraston, D. (2011) Creativity, Complexity and Reflective Practice, In Candy, L. and Edmonds, E. eds. Interacting: Art, Research and the Creative Practitioner, Libri Publishing Ltd. Oxford. (To be Published :Nov 2011. I should be able to upload a pre-publication PDF of this after the book launch end of Nov 2011)
Webpage for the book:
Book launch & performance:
Interacting: Art, Research and the Creative Practitioner
Edited by Linda Candy and Ernest Edmonds – Libri Publishing, UK
The Launch | Thursday 25 November @6pm – Queen Street Studios – 10-14 Kensington Street, Chippendale
***END NOV 2011 UPDATE***
This brief research report presents some demo sound files from a long wire instrument acoustic recording of a natural rainfall event in the recent large scale storms in NSW.
This recording was conducted as part of the Centre for Research in Complex Systems (CRiCS) Rainwire project at CSU. The Rainwire project forms part of an initiative to overcome critical shortcomings in existing land based natural rainfall measurement and analysis by investigating a long wire rainfall sensor system. It focuses on
i) signal processing challenges of these novel systems
ii) investigating high resolution temporal and spatial analysis of rainfall induced acoustic vibrations via long wire spans
iii) analysis of rainfall induced acoustic vibrations to improve detection, classification and quantification of fundamental natural rainfall events over land using a novel complex systems methodology.
Rainwire Project Background
A central environmental and climatic problem of 21st Century science is the protection of freshwater resources. Availability of freshwater for human consumption, agriculture and industry is both a national and international concern. The foremost source of freshwater is rainfall, and underground water sources are also ultimately dependant on this same source. The complex problem of understanding natural rainfall events is vital for informed sustainable land management, and fundamental research in complex systems, climatology and meteorology. The project will deliver national benefit by positioning Australia at the forefront of environmental sonification by demonstrating fundamentally different and novel approaches for land based rainfall research. Key algorithms will be developed for extracting the sound signatures of different rainfall patterns from induced vibrations on long wire spans. Human listeners are capable of identifying rainfall patterns acoustically with long wire sonification. As with face recognition by computer, many years of research will be required to match human faculties with machine algorithms. There is therefore great potential for future research into this aspect.
The Rainwire project will be a significant advance for an important area of non-linear vibration research in long wire physics. The long wire span is a suspended cable, a key complex dynamical system with applications in many fields of engineering (mechanical, civil, electrical, ocean and space). Suspended cables have significant research interest, in particular the investigation with stochastic / random excitation and rain-wind induced vibration, is a vital area where new studies and results are important (Rega 2004b, Ibrahim 2004).
Rainfall event properties are key requirements for research in environmental processes (e.g. canopy interception, flooding, soil erosion, run-off, overland flow, ponding), agricultural processes (e.g. wash-off of agrochemicals on plant foliage, water use efficiency, geochemical and nutrient balance), flood management, rainfall simulation and modelling, built environment and urban drainage. Research in understanding and detecting global and regional environmental change require these rain event properties to be analysed at the sub-daily level. Such high resolution rain event properties have even greater importance given the likelihood of an increase in extreme rainfall events and associated requirements for adaptive management.
Rainwire has the potential to provide new weather / climate based products and information services, potentially available in real time over the internet or through mobile devices. Information use would be “smarter” through improved rainfall data for agriculture, with potential for use in a number of other industries. A longer term future application is in general problems of irrigation. Gardens, parks, sports fields etc. are commonly seen being sprinkled immediately after heavy rainfall, sometimes during rain itself. The virtue of this technology is that it can be extended to operate across any wire fences. This in turn, with the increasing availability and decreasing cost of digital signal processing chips and wireless computer communications, makes it possible to move to much more controlled irrigation, saving precious water supplies.
The Rainwire project will have wide national and international significance due to its foundation in the field of complex systems research, which coupled with the unique approach of integrating rainfall sonification, represents the cutting edge of multidisciplinary scientific research. Better models of complex systems coupled with new mathematical and algorithmic tools, will eventually lead to benefits in human health, national security, economic well-being and sustainability.
Acoustic investigation of rainfall on land has been minimally investigated (Michaelides 2008). This stems from an inherent problem with designing acoustic sensors to cover a wide spatial area. Researchers have recently attempted to directly transfer underwater acoustic methods of rainfall analysis to small water tanks of <50cm2, but much more work is needed due to problems such as high maintenance requirements, tank freezing and difficulties with bubble noise in the small tank chamber (Winder and Paulson 2010). Current vibrating wire instruments for rainfall measurement are expensive and cover a very small (<1m2) geographic area (Duchon 2008), and as such are impractical for use on farming properties by land managers or for other key rainfall event properties. Other sensors such as optical disdrometers/distrometers and vibration based droplet spectrometers suffer from similar problems of expense and very small spatial area coverage (Michaelides 2008). The limitations for new sensors are imposed by a lack of signal processing methods, rather than technological or even cost issues.
Methods of sonification of environmental data for scientific application to date have been based on digital sound generation from data, as opposed to analogue means. In these projects the phenomena under examination have been sampled to create data sets that are subsequently ‘mapped’ in an arbitrary way to sound synthesis engine parameters that produce audio output (Childs & Pulkki, 2003). However, the more the data is mediated, the less direct the relationships are between the stimuli and responses. The resultant audio in typical sonification projects bears an arbitrary relationship to the source phenomena because the process is abstracted through the creation of a data set.
Rainwire will contribute to the complex systems research knowledge base in the following key areas:
i) extending the scope and methodology of acoustic rainfall detection, classification and quantification from its present use in underwater systems to a land based system. This will be enabled through the application of signal processing, and new/existing complexity measures.
ii) extending knowledge in the non-linear dynamics of random excitation and fluid interactions with suspended cables
iii) publically available datasets of high resolution long wire instrument rainfall sonifications for explorations of physical theory and pattern recognition.
Acoustic analysis using Digital Signal Processing (DSP) techniques have been successfully applied to high resolution rainfall measurement and analysis at sea using underwater acoustics for decades. Initial research was conducted during World War II when rainfall was discovered to impact on military sonar. Techniques were subsequently developed for Acoustic Rain Gauges (ARG) to identify rainfall events through unique frequency spectrum characteristics between 1 and 50kHz (Black et al 1997, Amitai and Nystuen 2008). The unique characteristics of rainfall impacting water are created by the initial impact and the subsequent formation of an underwater bubble for certain raindrop sizes. These variable drop impacts produce different frequency signatures as a result of this unique mechanism, which can be used to deduce important rainfall parameters. Rainfall sounds are recorded by hydrophones attached to ocean buoys from a depth of tens or hundred of metres, with the most recent experiments at depths of 2km. The listening radius for these underwater ARG’s is nearly three times their depth, giving a very wide spatial listening area. The development of ARG’s was initiated by the difficulties of deploying standard rain gauges on the ocean surface. However, because they are isolated out at sea for very long periods, continuous recording is not possible because they need to conserve power. ARG’s spend much of their time simply checking if rainfall is present at the surface at specified intervals and can miss out on the beginning of a rainfall event. ARG’s are capable of high temporal resolution recordings of rainfall data due to the acoustic nature of the information recorded.
Sonification is the presentation of data or information via sound, and the most well known scientific instruments in this field are the Geiger counter and Sonar (Kramer 1994, Hermann 2008). Long wire instruments fundamentally differ from existing data based sonification processes and rainfall measurement devices by generating sonic events directly from rainfall patterns in realtime. Sonfication from real world physical actions, as opposed to being mediated via electronic sound synthesis mapping, can be seen in an early example by Galileo Galilei in the formation of the law of falling bodies (Plessas et al 2007). In this experiment Galileo attached bells to an inclined plane in order to make his discovery.
Long wire instruments are made from standard high tensile fencing wire constructed in single or multiple spans across an area of the landscape. Piezoceramic transducers are used to convert mechanical vibrations caused by rainfall events into audio signals for measurement and analysis, effectively sonifying the rainfall patterns. Long wire instrument spans can range from tens to hundreds of metres, up to a total multispan length of several kilometres or more, usually supported by poles or attached to very large rocks. Spatial arrangements can typically be in the form of a single line, parallel lines, radial lines from a central point to compass points (e.g. NESW) or other geometric shapes. Long wire instruments can be constructed on flat land, across gulley’s, down hillsides, over complex terrain and sections of water.
Long wire spans are classed as suspended cables, which exhibit a complex variety of non-linear dynamical behaviours, and are an archetypal complex system of interest (Ibrahim 2004, Rega 2004a,b). Complex systems is an emerging multidisciplinary science developing new ways of researching large, highly intricate, dynamical systems in diverse areas such as biology, physics, social networks, socio-technological systems, socio-ecological systems, economics and the environment (Mitchell 2009, Norberg & Cumming 2008).
Rainwire is conceptually innovative in that the field of scientific data sonification has emerged very recently. However, unlike most existing sonification systems where sound/music is attached to waveforms, in our project the sonification is intrinsic. The conceptual innovation now lies in identifying distinctive sound patterns and relating them to particular types of rainfall event.
The Sputnik Wires long wire instrument was used for this experimental recording. The pickups were attached to the Lower Wire span (3.15mm diameter wire) as shown in Figs 1 to 3 below. A complete 12.25mm rainfall event was recorded, with the rainfall reading taken using WIRED Lab’s rain gauge, located about 40m from the long wire instrument.
Fig 1. Pre-amps and field recorder
Fig 2. Water tight and wind proofed for recording
Fig 3. Pickups attached to bottom wire
Fig 4. Spectrogram of short section (1.6 secs) – note the difference in shapes of the frequency curves
Fig 5. Close up of frequency curves
Conclusions / Future Work
Long wire instrument rainfall recordings have resulted in a wide range of unique audibly human recognisable features, which have been dependant on rainfall event properties e.g. duration, intensity, event profile and drop size. These unique sound properties can take the form of high frequency crackles/sizzling, high to low frequency swept zaps similar to the sounds produced by a sound synthesizer, metallic pings, and clicks, all of which exhibit dynamic amplitude and spectral characteristics depending on the rain type. The big challenge is to now translate the immense sophistication of human pattern recognition into automated computer algorithms.
The current underwater acoustic classifications of rain drop size described in Nystuen (2001) are : tiny (<0.8mm), small (0.8 – 1.2mm), medium (1.2 – 2.0mm), large (2.0 – 3.5mm) and very large (>3.5mm). The unique spectral sounds produced by individual raindrops impacting on the wire spans will need to be identified. Detection, analysis and quantification will be derived from the underwater acoustics methodology. This will be through the use of DSP techniques such as sound level magnitude, ratio of sound levels at different frequency bands and temporal variation of sound levels within observed frequency bands. In order to obtain these spectral characteristics methods such as digital filtering and Fourier transformations will be implemented. These techniques have been successfully applied in underwater acoustic rainfall research (Black et al 1997, Amitai and Nystuen 2008, Amitai et al 2007, Ma and Nystuen 2007, Nystuen 1996, 2001), but it should be noted that the physics of the two processes are completely different resulting in different spectral signatures for rain induced vibrations on wire/suspended cables and water surfaces.
Future research will therefore require the detection of new spectral signatures associated with long wire systems, as well as the identification of any potential background noise or tones, and identification of any potential limitations. As with underwater acoustics, background sounds will need to be identified and taken into account. A land based long wire instrument can potentially be subject to a number of unwanted sounds through the sensitive piezo transducers such as : insect collisions on the wire, spiders, birds (both collisions and perching), tree and leaf debris, wind noise and Aeolian tones, man made interference sounds such as electric cattle fences and radio/transmission beacons. For this project these sounds are not wanted and are therefore noise. Following this initial research on obtaining spectral signatures for different drop sizes the approach using natural rainfall will be :
i) Identify wind speed for quantifying wind induced Aeolian tones and background noises
ii) Develop acoustic inversion algorithm empirically to match rain gauge level measurements and obtain a drop size distribution and rainfall rate
Following this acoustic inversion procedure it is then possible to apply the results to three main issues :
i) Relationship between rainfall rate (R) and radar reflectivity (Z). Z-R is the main quantity desired by most researchers using rainfall radar. The benefit of obtaining this quantity is also the potential to use the acoustic technique to calibrate rainfall radars.
ii) Classification of rainfall types as convective (heavier rainfall with shorter space and timescales) or stratiform (light wide spread rainfall). Underwater acoustic techniques can classify two sub types of each type, and the methods are based on identifying distinguishable spectral characteristics
iii) Temporal analysis of rainfall to show intraminute variability. Few other rain gauges can show this type of detail, which is a unique advantage of the high resolution of acoustic techniques. Allows for discrimination of rapid onset or cessation of subcells within rainfall events
The complex systems methodologies will encompass techniques from non-linear time series analysis which are recently being used in rainfall research (Sivakumar et al 2006), though not on acoustic data. Autocorrelation function, correlation dimension, entropy, Lyapunov exponents and phase space reconstruction using embedding are well known tools for detecting and quantifying various manifestations of deterministic chaos (Sprott 2003). These chaos measures have been used widely in acoustics and sound signal analysis, particularly as they provide a useful measure for wind induced tones, harmonic and inharmonic relationships, and white noise (Fletcher et al 1990, Fletcher 2000, 2002, Monro & Pressing, 1998). These measures will be valuable for identifying non-rain sounds, as well as rain induced sounds. The underlying theory for these techniques is called delay embedding, which provides global information about time and amplitude behaviours of signals. The results of applying these techniques are to provide a compact multi-dimensional representation of sound signals, with information regarding signal dynamics and amount of order or disorder present.
Complimentary to chaotic analysis are complexity measures which can provide a measure of a system’s organisational complexity (structure, regularity, symmetry and pattern). Complexity measures are an important complimentary addition to quantifying degrees of randomness, because measures of randomness cannot measure the structure or organisation within a system.
This research was funded by the Charles Sturt University Competitive Grant Scheme and conducted in association with the Centre for Research in Complex Systems (CRiCS) under the project title Rainwire : Complexity measures of rainfall acoustics. (This project funding by CSU has now ended).
References / Bibliography
Amitai, E and Nystuen, J. A. 2008. Underwater acoustic measurements of rainfall. In Michaelides, Silas C (Ed.) 2008. Precipitation : Advances in Measurement, Estimation and Prediction Springer
Amitai, E., Nystuen, J. A., Anagnostou, E. N. and Anagnostou, M. N. 2007. Comparison of Deep Underwater Measurements and Radar Observations of Rainfall, IEEE Geoscience and Remote Sensing Letters, VOL. 4, NO. 3, pp406-410
Childs, E. & Pulkki, V. 2003. Using Multi-Channel Spatialization in Sonification: A Case Study with Meteorological Data. Proceedings of the 2003 International Conference on Auditory Display. pp. 192–195.
Duchon, C. E. 2008. Using vibrating-wire technology for precipitation measurements. In Michaelides, Silas C (Ed.) 2008. Precipitation : Advances in Measurement, Estimation and Prediction Springer
Fletcher, N.H., Perrin, R. and Legge, K.A. 1990. Nonlinearity and chaos in acoustics. Acoustics Australia 18, 9-13
Fletcher, N. H. 1999. The nonlinear physics of musical instruments. Rep. Prog. Phys. 62, 723-764
Fletcher, N.H. 2000. A class of chaotic bird calls? J. Acoust. Soc. Am. 108, 821-826
Fletcher, N. H. 2002. Harmonic? Anharmonic? Inharmonic? Am. J. Phys. 70, 1205-1207
Hermann, T. 2008. Taxonomy and definitions for sonification and auditory display. Proceedings of the 2008 International Conference on Auditory Display,Paris, France, June 24-27, 2008
Ibrahim, R. A. 2004. Nonlinear vibrations of suspended cables – Part III: Random excitation and interaction with fluid flow. Appl Mech Rev 57(6):515-549
Kramer, G. (Ed.) 1994. Auditory Display: Sonification, Audification, and Auditory Interfaces. Reading, MA, Addison Wesley Longman.
Ma, B. B. and Nystuen, J. A. 2007. Detection of Rainfall Events Using Underwater Passive Aquatic Sensors and Air–Sea Temperature Changes in the Tropical Pacific Ocean. Mon. Weather Rev. 135:pp 3599-3612
Michaelides, Silas C (Ed.) 2008. Precipitation : Advances in Measurement, Estimation and Prediction Springer
Mitchell, M. 2009. Complexity: A Guided Tour. New York: Oxford University Press.
Monro, G. & Pressing J. 1998. Sound Visualization: The Art and Science of Auditory Autocorrelation, Computer Music Journal, 22:2, pp. 20-34
Norberg, J. & Cumming, G. 2008. Complexity Theory for a Sustainable Future, Columbia University Press
Nystuen, J. A. 1996. Acoustical rainfall analysis: Rainfall drop size distribution using the underwater sound field, J. Atmos. Oceanic Technol., 13, 74-84
Nystuen. J. A. 2001. Listening to raindrops from underwater: An acoustic disdrometer. J. Atmos. Oceanic Technol., 18, 1640–1657.
Plessas, K. V. W., de Campo, A, Frauenberger, C. and Eckel, G.M. 2007. Sonification of spin models. Listening to phase transitions in the Ising and Potts model. Proceedings of the 2007 International Conference on Auditory Display. pp. 258-265
Rega, G. 2004a. Nonlinear vibrations of suspended cables – Part I: Modeling and analysis. Appl Mech Rev 57(6):443-478
Rega, G. 2004b. Nonlinear vibrations of suspended cables – Part II: Deterministic phenomena. Appl Mech Rev 57(6):479-514
Sivakumar, B., Wallender, W. W., Horwath, W. R., Mitchell, J.P., Prentice, S.E. and Joyce, B.A. 2006. Nonlinear analysis of rainfall dynamics in California’s Sacramento Valley. Hydrol. Process. 20, 1723–1736
Sprott, J. C. 2003. Chaos and Time-Series Analysis. Oxford University Press.
Winder, P. N. and Paulson, K. S 2010. An acoustic disdrometer: the measurement of rain kinetic energy and rain intensity using an acoustic disdrometer. 15th Symposium on Meteorological Observation and Instrumentation, American Meteorological Society.