ࡱ> ܥhc el09]-888888888888 8 8891899999999999999K9X9X9899999998898999989899px:8888889999Statistics Commission Talk: September 2006 Some Assorted Problems with Statistical and Economic data, and Their Implications Jim Cuthbert Margaret Cuthbert We would like to thank the Statistics Commission for inviting us to talk today. Since 1997, we have spent quite a bit of time researching in the general areas of the operation of devolution, public expenditure, financial accountability, and the Scottish economy. We have published around 40 articles, many highly statistical in content. As a result, we feel we have something useful to say on statistical development in Scotland. We see what we are going to say today as a positive contribution. Our talk will be structured around the answers to a sample of questions. We will discuss each of these questions briefly: but our main concern will be with the general implications we will draw at the end. For those wanting more detail on the individual questions, see papers on our website www.cuthbert1.pwp.blueyonder.co.uk . The questions, and brief answers, are as follows:- Is it possible to tell, from the published statistics, 1) The size of Scottish imports and exports by sector, and by country of origin or destination? No: first, Quarterly Regional Trade in Goods figures, a National Statistic, show Scottish imports from the EU as a % of UK imports at 2.2% and from non-EU countries as being 4.7%. These figures are clearly wrong, and indeed are regarded as such by the Scottish Executive, but no action has been taken to resolve the situation. Second, for several years, the trade figures for Scotland contained a substantial dustbin postcode for the whole of the UK which was sited in Aberdeen and held ghost imports and exports. The amounts were very large relative to Scottish recorded trade, and variable. Despite the statistics being received quarterly by several civil servants in the Scottish Executive, this was allowed to continue and distort all Scottish trade figures until we requested a detailed a breakdown of the published data, identified the problem, and asked for the offending postcode AB11 6GY to be removed from the Scottish data, which was duly carried out by Customs and Excise. 2) For the Scottish Executives flagship policy of free personal care for the elderly, how actual spending relates to the planned cost of the policy? No: in fact, published statistics purport to provide data on total expenditure on free personal care services to the elderly while the policy was introduced on the basis of the additional cost to local authorities of extending free personal care to all elderly. So even in principle published figures cannot be compared to the original policy costings. Second, the original costings were seriously flawed, having underestimated the size of the disabled elderly population by around 50%. Third, the published Scottish Executive statistics contain clear inaccuracies. To give one example: for Glasgow, by far the largest authority, Scottish Executive statistics show the cost of delivery of an hour of personal care in the home to be 5: when asked under Freedom of Information, Glasgow Council give the figure as being 9. 3) For Whitehall Departments, with a mix of devolved and reserved services, whether the way they spend their money corresponds properly to the geographical range of their responsibilities? No: but after a great deal of digging, you can, for example establish that 12.9 million of spend by DTI in 2002/03 on inward investment was for grants to the English Regional Development Agencies, with no corresponding Scottish spend: and yet this is classed as a reserved function of DTI. Just one of several significant examples we have uncovered where departmental spend does not correspond to the geographical range of their responsibilities. 4) Whether the figures for identifiable public expenditure in Scotland and England are on a comparable basis? As regards the identifiable expenditure comparison published in the Treasurys PESA, the 2005 publication said that figures for expenditure per head in the regions of England and the countries of the UK are therefore directly comparable. But when we obtained the detailed PESA database, by means of a Freedom of Information Act request, we found that at least 4.4 billion of expenditure, (on a range of services, including prisons), is excluded from the basis of identifiable expenditure in England, while expenditure on the same services is included in Scotland. At a sub-programme level, comparisons in some cases are nonsensical: if feeding into policy, they will give entirely the wrong message. As a result of our pointing this anomaly out, there is now a rather limited acknowledgement of the point in PESA 2006, and a quite unsatisfactory proposal for dealing with the difficulty. 5) For those services devolved to Scotland, what expenditure is in England on the same services? No: because subprogramme data in the PESA database does not hold a coding for whether the expenditure is reserved or devolved: and if you try to do it for yourself, (using the Treasury Funding Statement, and trying to link this with PESA database obtained under Freedom of Information), you find that it is impossible to align sub-programmes between the two publications. 6) How much provision has been transferred out of the public expenditure programme for water in Scotland for the period 2002-2010? No: and if you find this difficult to believe, then read the paper to the Finance Committee by its adviser, Arthur Midwinter, where he struggles to answer this question for 2002-06: in fact, over 2002-10, at least 500million has been transferred out of the water budget, though you would never recognise this without probing deep into published information from a number of sources. These three questions raise in particular issues of data access, data accuracy, database design, financial accountability, and what we might describe as, having an appropriate data culture. So, as we have seen, the answer to each of these questions is essentially no. We could go on, listing similar questions relating to EU receipts: EU grants: higher education costs: incapacity benefit etc: but enough is enough on the specifics. What does all this tell us in general terms. First of all, there is a need for government to change its culture, so that much more detail is published. Each of the above anomalies only became clear when we obtained detailed information, sometimes under Freedom of Information, and started to look at the credibility of disaggregated information. If all you publish or make available is aggregated information, anything can be, and probably is, lurking in the woodwork. Secondly, as regards Public Expenditure statistics, the Treasury really needs to give itself a shake. The following two quotations illustrate this. From Professor David Heald: The Treasury takes a proprietary view of its database and denies access to this even to the pre-devolution territorial departments and the post-devolution Executives. This illustrates a culture of secrecy in the Treasury which is quite out place today. The second quotation is from the Treasury itself, (in response to our suggestion that figures should be produced for expenditure in England on services devolved in Scotland): It would be difficult to produce such a table. The extent of devolution varies in the three countries. Data structures have not been designed to allow such comparisons so estimates might be needed, and it might not be easy to get these right enough to be of publishable quality. Moreover, any results would need to be interpreted with care. Policies are increasingly diverging under devolution, so there may not be a direct read across. This last point is precisely the reason why the analyses we are suggesting are essential. But most tellingly, the earlier point, that data structures have not been designed appropriately, illustrates a significant cultural problem that appears to exist in the Treasury. Their first thought, on being confronted with the major changes implied by devolution, should surely have been to consider what coding changes were needed to the PESA database: and this should certainly have involved considering including for each item of expenditure a code showing devolved/reserved status, and a code for non-identifiable expenditure showing over what geographical level the expenditure was non-identifiable. But instead of approaching things this way, the Treasury culture seems to be concerned with presentational issues first, and data-structure issues second, or not at all: (one could speculate that this culture may well come from the Treasury being a department where there are too many economists). Thirdly, as regards government in Scotland, there is another problem. Consider, for example, the following . First is written evidence to the Scottish parliaments Audit Committee in 2004, by the then head of the Scottish Executives Health Department. In that he listed all the quality checks on the data and stressed how the department went back to check inconsistencies with local authorities. This does not explain how published figures come to contain the kind of inaccuracies we have highlighted above, with, for example, the cost of delivery of one hour of personal care being less than the minimum wage in Scotlands largest local authority. The following quotation is from the Chief Executive of the Scottish Parliament, in response to a letter we sent him alerting him to the problems with the Customs and Excise trade data for Scotland, which the Parliament purchases:- although there are some flaws in the Customs and Excise figures, they are nevertheless the best available at the current time. They are also the official, established and most widely referred to statistics (by the academic and business community), and it would be odd if our Members were denied access to them. Or on behalf of Jack McConnell himself, when we alerted him to the fundamental problems with the costing of the free personal care policy: the Care Development Groups report set out careful calculations and the reasoning behind them. Thank you for your interest in our work. The above quotations are indicative of an attitude towards data quality which we can only say is too complacent. The Executive has to be clear what it is necessary to measure, has to get into a culture of making much more detail available: and has to make greater efforts to ensure that the detailed level data is correct and plausible. But finally, there are also implications, not just for government in Scotland, but also for the Scottish public generally. It could be said that we suffer in Scotland from a particular manifestation of the Scottish cringe, in the form of deference to perceived authority. To an unfortunate extent, truth, in the Scottish economo-political debate, is not defined as something which is independently verifiable: but is whatever perceived authority, in the shape of official statistics, or certain professors, or certain parts of the media, states it to be. This is a sad come-down from the days of independent Scottish thought, as epitomised by David Hume, (a name now sadly hi-jacked by the establishment). A more healthy attitude would be to regard as truth that which can, if necessary, be verified by oneself from plausible disaggregated data: and to regard all else as not proven. But we can only adopt this principle if we demand, and get, access to much more detail than is available at present: and if we have the appetite and skills to use that detail. PAGE 1 .A}~ AC~  R W LMP 8' r~$sy/!% %()Q**1,<,[0\0]0^0d0e0f0g0h0k0l00uPaP uDPV^U?+,~  C~   LM 9OPqry 0!%%%%',''())**0,1,Y0Z0[0\0]0h0i0j0k0l0`K@Normala c"A@"Default Paragraph Font@TOC 1a  @ Footer 9r )@ Page Numberl-l0 0'l0l- !@1Times New Roman Symbol "Arial"h"&e,Fg%OX*Statistics Commission Talk: September 2006Preferred Customercuthbert !)Root Entry Fpx: WordDocument9CompObjjSummaryInformation(  FMicrosoft Word Document MSWordDocWord.Document.89qOh+'0$ 8D l x  +Statistics Commission Talk: September 2006e8DPreferred CustomerNormal cuthbert1MicrDocumentSummaryInformation8   FMicrosoft Word Document MSWordDocWord.Document.89qDell Computer Corporation;O +Statistics Commission Talk: September 2006osoft Word for Windows 95@@q @v@Ip:g%՜.+,0@Hlt | Dell Computer Corporation;O +Statistics Commission Talk: September 2006