Random Uncertainty in Water Balances

Assessing Random Uncertainty in

Water Balance Calculations

Getting started – a practical approach

How reliable are volumes of Non-Revenue Water and its components, calculated from a Water Balance? The answer depends on many factors and varies but it can be assessed. Practitioners who understand how to do this can quickly identify which water balance components should be targeted as priorities to reduce random uncertainties to an acceptable level, depending on local circumstances and whether the purpose is a preliminary assessment or a more detailed audit.

This paper seeks to de-mystify this topic by minimising the use of specialist statistical terminology and explaining, step by step, how to set up basic spreadsheet calculations. It is hoped that practitioners without a deep knowledge of statistics can quickly gain important insights into assessing the uncertainties of water balance calculations, carrying out sensitivity testing, and quickly identifying priorities to improve the reliability of calculated NRW, Apparent and Real Losses volumes.

In the IWA Water Balance shown in Figure 1 below, a clear distinction is made between System Input Volume (which may include Water Exported to another distribution system) and Water Supplied (which excludes Water Exported). Each component of volume entered in a water balance may have a systematic error (either always positive, or always negative). Systematic errors in bulk metering, which are particularly influential in generating substantial errors in assessment of NRW and its components, are discussed in detail by Johnson (2016). The Water Balance in Figure 1 clearly states that volumes have been corrected for known systematic bulk metering errors.

Figure 1: An IWA Standard Water Balance corrected for known systematic bulk metering errors

wloss 1

Even after known systematic errors in bulk meters have been corrected, these and other components of the Water Balance will also have random errors (equally likely to be positive or negative). This paper outlines a practical approach to assessing the combined influence of these random errors on the uncertainty of the calculated volumes of NRW, Apparent Losses and Real Losses. The mathematics and terminology of the so-called Normal (bell-shaped) probability distribution of errors, which is often used for these calculations, can be intimidating to practitioners who have not studied statistics, so they are frequently reluctant to attempt such calculations. Henry Ford said ‘Whether you think you can, or think you can’t, you are probably right’. This paper is written to encourage those who think they can, or are at least prepared to try!

Table 1 shows an example IWA Water Balance calculation of Figure 1 in tabular form. Volumes in Million cubic metres (Mm³) have been entered in the yellow cells in Column 3 on Rows A, B, D, F, G and I. Volumes in pink cells in column 3 have been calculated.

For the purpose of simplifying this demonstration of how to calculate uncertainty, a practical approach has been used to assess the mainly unmetered components of NRW in Row L (Unbilled Authorised Consumption), Row N (Unauthorised Consumption) and Row O (Customer Metering Errors). The %s in the blue cells in Column 2 are maximum guideline values expressed as % of Billed Metered Consumption (Row H), as recommended in Table 3 of the EU Reference Document (2015), for a direct pressure system without roof tanks, in a high income European country. These maximum guideline values for initial calculations can of course be increased or decreased if appropriate for individual country circumstances; examples are given in Johnson 2016).

Table 1: Example Water Balance used for the purposes of this Guidelinewloss 2

In this example, the water balance shows NRW as 1.900 Mm³, Water Losses as 1.861 Mm³, Apparent Losses as 0.164 Mm³ and Real Losses as 1.697 Mm³. However, how can we assess the uncertainty associated with these calculated figures?

Part 1 of this paper explains how the uncertainty of calculated NRW, Water Losses, Apparent Losses and Real Losses can be calculated, step by step, based on the assessed uncertainties in the metered and assessed volumes entered into the calculation.

Parts 2 to 4 explain the basic mechanism of calculations of Uncertainty when adding or subtracting components of the water balance, with a few ‘Check your understanding’ questions. These take the calculations as far as Row E (Water Supplied).

Section 5 then shows the overview mechanism of calculations for the whole Water Balance, with the detail of the calculations from Rows E (Water Supplied) to Row Q (Real Losses) set out in detail in Appendix.

Note that although uncertainty in +/- % terms is used as part of the assessment calculation, it is preferable to ultimately quote uncertainty in volume terms which tend to be more stable than uncertainty in % terms.

Part 1: Introducing Uncertainty into Water Balance Calculations

Each of the measured or assessed volumes in a Water Balance has random uncertainties caused by broad combinations of different factors; some are large, others are small. This Part 1 of the paper explains how uncertainty is expressed (sometimes in +/-%, and sometimes in +/- volume terms) and shows the step by step calculations of uncertainty for the System Input Volume and Water Supplied components of the Water Balance.

The uncertainties of each component of volume can be expressed as:

  • Percentages of each of the volume components A,B,C….  (+/-ua%, +/-ub%, +/-uc%  etc. ), or
  • units of volume (+/-Ua, +/-Ub, +/- Uc …)

Because results of testing of bulk meters and customer meters are usually reported as percentages of recorded metered consumption, uncertainties of other water balance components, based on audits and personal judgement, also tend to be initially expressed as +/-%s of the component concerned. It is therefore usual to initially enter uncertainty figures as +/-%s, then to calculate uncertainty in +/-volume terms.

At this stage, the reader may consider that major investigations will be needed to assess detailed values for +/-u% for all Water Balance components, and become discouraged. However, remember that the first few water balances for any system usually contain several ‘best estimates’ for volumes, which are gradually improved over time. Therefore, apply the same principle to your initial estimates of uncertainty; have faith in your judgement, get started and improve as you progress.

Put together whatever knowledge you currently have, or could obtain fairly easily through asking direct questions and seeking opinions of water Utility personnel who provided the data. Inspect bulk meter installations, data transfer systems and data collection procedures. Then say to yourself ‘my initial assessment is that the +/-% uncertainty I should use for this component (say, Water Imported) is probably around +/- 2% of the likely true value’. You can always do sensitivity testing later, using improved estimates, after you have identified and prioritised investigations into what are likely to be the most influential components.

Part 2: Calculating Uncertainty for Water from Own Sources WOS

To show how to calculate uncertainty, start with the ‘Water from Own Sources WOS’, Row A in Figure 1. For the purposes of demonstrating the calculation process, in the example WOS is assumed to have three input components with different volumes and different +/-% random uncertainties.

Treatment Works X output meter, with the largest output volume, has been checked by an in-situ flow reference standard (see Section 8 of Johnson, 2016) and found to be within the random uncertainty of +/-2% for the primary (meter installation) and secondary (electronics etc.) devices, for the method of testing used. Random uncertainties associated with data collection for Treatment Works X output are assumed to be zero as the Supply Operations Centre is located at Treatment Works X. The assumed random uncertainty of +/-2% is close to the lowest practically achievable limit for a large bulk meter.

Treatment Works Y output meter, with the second largest volume output, is assigned slightly greater random uncertainty (+/- 2.5%) which includes an allowance for data collection errors . The volume pumped from standby borehole Z, with lowest volume output, is assessed from hand written records of pump running hours, and so the assumed random uncertainty is higher (+/-6%). Table 2 shows the calculation process, with arrows to indicate the steps of the calculations

Table 2: Assessing Random Uncertainty of Water from Own Sources WOS

wloss 3

For each component enter the volumes in Mm³ for A1, A2, A3, and calculate the sum (8.000). Then, for each component on rows A1, A2, A3, multiply the Volume (Mm³) by the % uncertainty (u%) to get the Uncertainty U in Mm³ as a volume for each source.

Next calculate U² for each component A1 to A3, then add all of the U² values (= 0.0148). Then calculate the random uncertainty U in Mm³ on Row A for the whole of the Water from Own Sources by taking the square root of 0.0148 (= 0.122). The random uncertainty as a % is then 0.122/8.000 = +/-1.52%

It now becomes easy to identify priorities for reducing the uncertainty of the total WOS component. Highest priorities have the highest Uncertainty Volume U (or the highest U², the resulting priorities are the same). Priority rankings 1, 2, 3 etc.  are calculated automatically in the final water balance in Tables 5 and 6, to facilitate sensitivity testing for all volume components input to the water balance.

Note that the random % uncertainty (+/- u %) of 1.52% for the total WOS volume is less than any of the individual random % uncertainties (2.0% to 6.0%) for the components of WOS. In general, the more bulk metering inputs there are to a system (from WOS and Water Imported), the lower the weighted random uncertainty % will be.

Figure 2 shows how, in a system supplied by N  sources with equal output and equal % uncertainty of u% ranging from +/- 6% each, to +/- 2% each, the overall weighted % random uncertainty for total Water from Own Systems can be calculated approximately as +/- u% divided by the square root of N. Section 3 of Johnson 2016 shows the additional influence of Systematic errors.

Figure 2: Random Uncertainty % reduces with multiple sources wloss 4

Part 3: Calculating Uncertainty for System Input Volume

You can now use the methodology shown in Part 2 to assess the uncertainty of System Input Volume, which includes bulk Water Imported. As payment is usually required for Bulk Water Imported, a low uncertainty of +/- 2% for one bulk import meter will be assumed.

Table 3 shows the calculation of uncertainty for System Input Volume SIV, which is the sum of ‘Water from Own Sources WOS’ and ‘Water Imported WI’. As expected, an increase in the number of inputs reduces the % uncertainty, from +/- 1.52% for WOS, to 1.21% for SIV; even though the % uncertainty for Water Imported (+/- 2.0%) is greater than the % uncertainty for Water from Own Sources (+/- 1.52%).

Table 3: Assessing Random Uncertainty of System Input Volume SIV

tab 3 system input volume new

Check your progress:  If there were 2 bulk import meters, each with input of 2.00 Mm3 and +/- 2.0% uncertainty, how would the random uncertainty of +/- 2.00% change?

Answer: use Figure 2: if the uncertainty for 1 meter is +/- 2.0%, the uncertainty for the sum of 2 meters with equal throughput is +/- 2.0%/SQRT (2) = +/- 1.41%

Draft a spreadsheet: Reproduce Table 3 as a spreadsheet, and substitute uncertainty of +/- 1.41% for Water Imported. What does the uncertainty of System Input Volume change to?

Answer: from +/- 1.21% to +/- 1.12%

Part 4: Calculating Uncertainty for Water Supplied

The methodology shown in Parts 2 and 3 can be used to assess the uncertainty of Water Supplied, which equals System Input Volume minus bulk Water Exported. As payment is usually required for Bulk Water Exported, we will assume one bulk export meter with volume recorded with an uncertainty of +/- 2%.

Note that in this calculation (Table 4), even though Water Supplied WS in Mm3 is calculated as one Volume (SIV) minus another, the values of U2 in the final column are always added to each other. In calculation steps such as this, in which some flow volumes are subtracted, the Water Supplied volume (Row E) is usually less than the System Input Volume (Row C). The effect of these changes is that the random % uncertainty in Water Supplied (+/-1.51%) is now greater than the random % uncertainty in System Input Volume (1.21%).

Table 4: Assessing Random Uncertainty of Water Supplied WS

water supplied new

 

Draft a spreadsheet: Reproduce Table 4 as a spreadsheet, and experiment by increasing the Water Exported in 1 Mm3 steps up to 11.0 Mm3. What happens to the Uncertainty of Water Supplied?

Answer: Uncertainty rises from +/- 0.151 to +/- 0.264 Mm3, and from +/- 1.51% to +/- 26.4%

Any system with a high prportion of Water Exported is likely to experience greater uncertainty in Water Supplied and subsequent water balance calculations than a similar system which does not export water. Transmission mains systems, where water exported is often a very high proportion of System Input volume, can experience extremely high uncertainty for real losses calculations.

Perhaps surprisingly, this problem is not generally appreciated unless random uncertainty is routinely calculated for distribution systems with substantial exports. Also, in Utilities with relatively low leakage (Infrastructure Leakage Index less than 1.5), several examples have been found, in supply Zones linked by a transmission main, where an unusually low ILIs in one Zone coupled with an unusually high ILI in the adjacent Zone have identified a systematic error in bulk metering of exports and imports.

Part 5: Water Balance with Uncertainty Analysis, showing calculations and Priorities

The step-by-step calculation methods described in Parts 2 to 4 contain the building blocks which have been used to progress to Row E (Water Supplied) in the Water Balance spreadsheet in Table 5 below. The same step-by-step approach has been used to complete the calculations. Readers wishing to follow and reproduce (in their own spreadsheets) can find the detailed calculations from Rows F (Billed Metered Consumption) to Q (Real Losses) in the Appendix to this paper.

This method of assessing uncertainty in Water Balances has been used for more than 15 years by the Author and a number of colleagues in the IWA Water Loss Specialist Group. It is often an unwelcome surprise for Utilities to realise that, in Water Balance calculations, the calculated volume of Non-Revenue Water and Real Losses can have significant random uncertainties even after known systematic uncertainties have been identified and allowed for.

However, because the calculation method automatically identifies priorities to reduce the uncertainty, available resources can be quickly targeted to the most appropriate activities.

Table 5: Water Balance with Uncertainty Analysis, showing calculations and Priorities

wloss 7

A more user-friendly presentation of the Table 5 calculations is shown in Table 6.

Readers may have assumed that components with the largest +/- % uncertainty would be the priorities for investigation, but Table 6 shows that it is the multiple of volume x % uncertainty which defines priorities for reduction of random uncertainty in calculated NRW and Real Losses.

Table 6: User-Friendly Water Balance with Uncertainty Analysis, with U² calculations hidden

wloss 8

The author encourages readers of this paper to create their own spreadsheet to reproduce the calculations in Tables 5 and 6, and then to do ‘what-if’ sensitivity testing by increasing or decreasing the assumed uncertainty in the yellow cells to see the effect on overall uncertainty. Priorities in Column 7 to reduce uncertainty are calculated using the U Mm³ values in Column 6. Anyone who has seriously attempted to create such a spreadsheet and is experiencing problems is invited to Contact Us.

It is usually the case that bulk metering and billed metered consumption will contribute most strongly to the uncertainty for direct pressure systems. However, for systems where customers have individual storage tanks, uncertainty for under-registration of metered consumption can be a major component of uncertainty

Quoting volume and uncertainty of NRW and its components to four significant figures implies an unjustified degree of precision. The uncertainty assessment should be considered a part of the audit process which can be summarised as follows: ‘Uncertainty calculations using reasonably realistic assumptions have been used, with sensitivity testing, to identify and prioritise checking of water balance components which contribute most to uncertainty. Resulting uncertainty of individual components has been reduced to ‘fit for purpose’ achievable values which are consistent with assessed Real Losses of 1.70 Mm3 with assessed uncertainty of +/- 0.20 Mm3.’

Summary and Further Initiatives on Uncertainty in Calculating Real Losses

  1. The author and some international colleagues have been using these techniques, with more or less statistical detail, for over 20 years, to give practical insights into the reliability of Water Balance calculations, and to identify how to prioritise components of a water balance for further investigation, and to reduce the uncertainty of the calculations.
  2. The author believes that many practitioners could benefit from being able to undertake basic uncertainty analysis of water balance calculations, but are deterred from doing so due to lack of understanding of what appear to be complex statistical methods.
  3. This paper presents a basic step-by-step methodology to encourage others to overcome this problem and create their own spreadsheet calculations, in the interests of improving the reliability of international calculations of NRW and real losses.
  4. By showing that uncertainty calculations for water balances are basically just a repetitive step-by-step process with simple rules, rather than an intimidating calculation which only a statistician could understand, it is hoped this paper will encourage more leakage practitioners to begin to incorporate uncertainty assessments into their Water Balances.
  5. The methodology described here only applies when all the parameters (in this case, volumes) are in the same units. If there is sufficient interest – use Contact Us for feedback -further Guidance Notes may be added to the LEAKSSuite website to show step-by-step practical methods for calculations of random uncertainty in performance indicators and in night flow analysis.

Allan Lambert        20th May 2016

Appendix: Assessing Random Uncertainty for Components Water Supplied in the Water Balance

A1 Uncertainty for Billed Metered and Unmetered Consumption (excluding Water Exported)

The calculation in Table A1 shows the assessment of uncertainty for Billed Metered and Unmetered Consumption. As Residential and Non-Residential meters may be read at different frequencies, and Non-Residential Meters may have different under-registration characteristics when assessing Apparent Losses, most Water Balances separate Billed Metered Residential and Billed Metered Non-Residential consumption.

Table A1 initially adds Billed Residential and Billed Non-Residential metered consumption (Rows F and G), to obtain the uncertainty of Billed Metered Consumption BMC on Row H. It is assumed that the % uncertainty of metered volumes entered in Rows F and G are +/-2%; this allows for a robust customer meter data collection system with meter readings at least twice a year, substitution of missing data, and correction for influences of meter lag, where the metered volumes are adjusted to be equivalent to the period between the start and finish dates of the Water Balance. In some cases, where there are 5 or more separate categories of Billed Metered Consumption, it is hoped that the reader may now be confident enough to attempt to create an appropriately detailed uncertainty assessment calculation in their own spreadsheets, and to split Customer Metering Errors into the same number of components.

The uncertainty of assessed Billed Unmetered Consumption volume on Row I is assumed to be +/-20% for the initial uncertainty calculation. This can be adjusted later for sensitivity testing. Billed Unmetered Consumption (Row I) is then added to Billed Metered Consumption (Row H) to get the total Billed Metered and Unmetered Consumption volume, and its uncertainty, on Row J.

Table A1: Assessing Uncertainty for Billed Metered and Unmetered Consumption

(excluding Water Exported)

new tab 1

In this example, the addition of Billed Unmetered Consumption with a much higher % uncertainty than Rows F, G and H causes the uncertainty for Row J to rise, compared to Row H.

A2 Uncertainty calculation for Non-Revenue Water

Non-Revenue Water is Water Supplied (Row E from Table 4) minus Billed Metered and Unmetered Consumption (Row J, calculated in Table A1 above). The calculations are shown in Table A2.

Table A2: Assessing Uncertainty for Non-Revenue Water

new tab 2

A3 Uncertainty calculation for Water Losses

Water Losses WL (equal to Apparent Losses + Real Losses) are calculated by deducting Unbilled Authorised Consumption UAC from Non-Revenue Water. The maximum guideline value for UAC recommended in Table 3 of the EU Reference Document (2015), for a direct pressure system without roof tanks, is 0.5% of Billed Metered Consumption. As this is usually mostly an unmetered component the uncertainty u is assessed as +/- 20%.

Table A3: Assessing Uncertainty for Water Losses

new tab 3

A4 Uncertainty calculation for Apparent Losses

Apparent Losses (AL) are the sum of Unauthorised Consumption (UC) and Customer Metering Errors (CME). The maximum guideline values for UC and CME recommended in Table 3 of the EU Reference Document (2015), for a direct pressure system without roof tanks, are respectively 0.1% and 2.0% of Billed Metered Consumption. As unauthorised consumption is an unmetered component the uncertainty u is assessed as +/- 20%. Uncertainty of Customer Metering errors can be assessed from meter testing programs, but in the absence of such programs for this system, uncertainty is assessed as +/- 20%. That is to say, +/- 20% of the default of +/- 2.0% of Billed Metered Consumption, so the Customer Metering Error is between +/- 1.6% and +/- 2.4%.

Table A4: Assessing Uncertainty for Apparent Losses

new tab 4

A5 Uncertainty calculation for Real Losses

Real Losses (RL) are calculated as Water Losses (WL, calculated in Table A3) minus Apparent Losses (AL, calculated in Table A4). The calculations are shown in Table A5.

Table A5: Assessing Uncertainty for Real Losses

new tab 5