Getting the number right

By Jyoti Rahman. 

Published in the Daily Star on 11 June 2010

This piece explores the accuracy of Bangladesh government’s economic outlook. 

 

LAST night’s budget is predicated on the real economy growing by 6.7 per cent in the financial year 2010-11 (FY11). Is this macro-economic outlook too optimistic? What do international agencies such as the ADB or the IMF forecast? Whose forecasts are on average more accurate? How would we know about the outcome anyway? How reliable are the GDP estimates published by the Bangladesh Bureau of Statistics? Over the coming weeks, there will be much discussion about the budget. It’s important that these discussions are informed by answers to above questions. 

Let’s start with an analysis of whether budget’s macro-economic outlooks have a bias. In Chart 1, the horizontal axis represents the budget outlook for GDP growth in each financial year since FY03, while the vertical axis represents the actual GDP growth recorded in each year according to the latest National Accounts. 

The chart is divided by a 45 degree line. If the actual outcome in a given year is lower than the budget outlook for that year (that is, the budget was based on an optimistic outlook), then it lies below the 45 degree line. For example, the budget for FY08 was based on GDP growth of 7 per cent, but the actual growth outcome for the year was 6.2 per cent. For the past four years, budget has been based on an optimistic outlook — the impacts of domestic political instability and the global financial crisis have been consistently underestimated. 

How do these forecasts stack up against those of the ADB and the IMF? 

At the time of budget for FY09-10, both the ADB and the IMF were forecasting GDP growth of 5.2 per cent for the financial year — significantly more pessimistic than the government. While the ADB forecasts had been within the 5 per cent margin of error of the budget outlooks in six of the past eight years; the IMF has been consistently more pessimistic than the government since FY06 (Chart 2). 

 

(Top left) The initial growth forecast for FY10 was 5.5 per cent, but it was subsequently revised up to 6 per cent; FY10 actual is provisional BBS estimate. Source: Ministry of Finance, Bangladesh Bureau of Statistics. (Top Right) Source: Bangladesh Bureau of Statistics. (Bottom) The columns represent actual GDP growth; squares represent budget outlook, with a 5 per cent margin of error; triangles are the ADB forecasts; dashes are the implied IMF forecasts for financial years. Source: Ministry of Finance, Bangladesh Bureau of Statistics; ADB; IMF; author’s calculation. 

Using common measures of forecasting performance, a more rigorous analysis of forecasting performance of the government, the ADB and the IMF is presented in Table 1. 

  

 

  

Table 1: Forecasting Performance
  Budget ADB IMF
Mean error 0.15 0.02 -0.24
Mean absolute percentage error 0.46 0.49 0.45

The first row in the table is mean error, which measures the bias of the forecasts — a positive number indicates that, on average, the forecast tended to be larger than the outcome (overestimation bias) while a negative number indicates forecasts tended to be smaller than the outcome (underestimation bias). 

The mean absolute percentage error is in the second row, which measures the accuracy of the forecasts, as it measures the average distance between the forecast and the outcome. For both bias and accuracy, a number closer to zero indicates a better forecasting performance. 

It is clear that budget government forecasts do have an overestimation bias, while the IMF has a bigger underestimation bias. However, in terms of accuracy, there is not much difference between the three forecasts. 

Thus, if the recent history is any guide, last night’s budget is based on an optimistic outlook, but this outlook is, on average, just as accurate as major multi-lateral agencies. 

The above analysis, of course, is based on the final GDP outcome as published by the BBS National Accounts. How good are these numbers? Every summer, as the budget is produced, the BBS publishes an initial, provisional, GDP estimate using data from the first three quarters of the financial year. 

This provisional estimate is then revised the following year, by which time complete data about production, income, and expenditure are available. Obviously, there might be sound reasons for the revised, final, outcome to be different from the initial, provisional, estimate. Ideally, there is no reason why these revisions should be biased in any particular direction. 

It turns out, however, that the BBS estimates have a small overestimation bias. In 12 of the past 15 years, the BBS had revised GDP downwards, on average by 0.06 of a percentage point. 

This is shown in Chart 3 — here, the horizontal axis is the provisional estimate, the vertical axis is the final outcome, and overestimated years are below the 45 degree line. 

There is, however, more to the tale. The BBS estimates real GDP with 1995-96 as the base year. Given the structural changes in the economy, a rebasing is surely warranted. Bangladesh is also the only major economy in the region that does not produce quarterly national accounts. Severe resources shortage in the BBS is a problem too. 

And then there is the potential political interference. For example, the biggest downward revision was by 0.77 of a percentage point for FY01 — that was the year when Bangladesh’s GDP growth touched 6 per cent mark for the first time, and downward revision was done after a new government was elected. Given the sorry state of our institutions, what incentives do the statisticians have to produce an accurate number, as opposed to a number that may be politically convenient? What would the statisticians do when they are openly rebuked by powerful ministers for not producing data that show government policies are working? 

For years, Greece fudged its books. Eventually the day of reckoning came. As we debate this and that provision of the Budget, it’s important that we have the right numbers. Otherwise, a Greek tragedy could befall us too.

Advertisements

Comments are closed.