A Quarterly Information Bulletin from the Methodology Division
USE OF ARIMA MODELLING TO IMPROVE SEASONALLY ADJUSTED ESTIMATES
REVIEW OF ANALYSIS BRANCH
SAMPLE AND FRAME MAINTENANCE THROUGH COMPUTER ASSISTED TELEPHONE INTERVIEWING
IMPLEMENTATION OF GENERALISED REGRESSION (GREG) ESTIMATION FOR THE RETAIL BUSINESS SURVEY ANALYSIS
Use of ARIMA Modelling to Improve Seasonally Adjusted Estimates
The Australian Bureau of Statistics (ABS) publishes seasonally adjusted and trend estimates which are calculated from original estimates using the ABS seasonal adjustment package, SEASABS.
The original estimates comprise seasonal, trend and irregular. The seasonal component represents the systematic and calendar related effects, the trend component represents the underlying direction in the series, and the irregular component is the remaining component of the series after the seasonal and trend components have been removed. Seasonally adjusted estimates are calculated by estimating and removing the systematic and calendar related effects from the original estimates. The trend is calculated by smoothing the seasonally adjusted estimates. Both seasonally adjusted and trend estimates need the unknown future data to achieve good estimation properties.
Both estimates are subject to revision at the current end of a series once additional observations become available or the original estimates are revised. An objective is to achieve accurate seasonally adjusted and trend estimates with small revisions at the current end of the series. This can be achieved using an Integrated Autoregressive Moving Average (ARIMA) modelling approach.
This will be implemented for the Monthly Retail Trade series (ABS Catalogue 8501.0) from August 2004. This method will be progressively introduced to other ABS time series as appropriate.
The current ABS seasonal adjustment approach, SEASABS, uses moving averages which are a set of weights of a given length designed to extract specific information from the data. As part of intermediate calculations in the seasonal adjustment process, the moving averages require data points to be available either side of the point of interest. At the end of a time series, assumptions are made about the data points that extend beyond the source series observations. These assumptions are built into the design of the asymmetric moving average weights used in the seasonal and trend moving averages. ARIMA modelling is a technique that can be used to extend estimates at the end of a series. The use of these techniques generally results in a reduction in revisions to data when subsequent observations become available.
The data points projected by the ARIMA model are temporary intermediate values that are only used to improve the estimation of the seasonal factors. The projected data do not affect the original estimates and are discarded at the end of the seasonal adjustment process.
Previous work (see references below) has evaluated ARIMA modelling against 820 series across 11 subject matters. It highlighted a reduction in the size of the revisions to seasonally adjusted estimates in the order of 7.7%. While ARIMA modelling on average reduces the revisions for each subject matter area, there can be individual series where the technique actually increases the size of the revisions. However, the overall average improvement and the sound theoretical underpinnings outweigh the chance of increased revisions in most series.
More details on the implementation of ARIMA models as part of the ABS seasonal adjustment process can be obtained in a forth coming issue of Australian Economic Indicators, Catalogue number 1350.0, or the Methodology Advisory Committee paper 'Use of ARIMA Models for Improving Revisions of X-11 Seasonal Adjustment', November 2001, available from www.abs.gov.au.
For more information, please contact Craig McLaren on (02) 6252 6540
Email: firstname.lastname@example.org or
Mark Zhang on (02) 6252 5132.
Review of Analysis Branch
Analysis Branch was formed in 2000, building on the functions which had been established 4 years earlier within MD at section level. The formation of the branch signalled ABS desire to significantly increase its analytical capacity.
The main focus of the branch has been to undertake projects that:
- build new statistical products;
- validate and improve on existing methods; or
- add value to data through the use of more complex analytical techniques.
The purpose of the review undertaken over the past few months was to define the focus of the branch's work over the next three years in the context of its agreed role and the experience of the last three years.
The review has looked at the strengths and weaknesses of the branch as well as threats and opportunities. It has also examined the characteristics of successful projects and how might these be applied in areas where projects have been less successful. Information was gathered from a wide range of clients in the ABS.
The review noted the significant achievements of Analysis Branch over the years. The major issues identified were:
- ensuring that ASB projects are aligned with ABS strategic priorities for economic and social statistics the ownership of ASB projects by subject areas;
- ensuring projects are a joint product of ASB and the subject area;
- ensuring that ASB and subject staff have the skills required to undertake projects;
- ensuring that ASB projects have a tangible and useful outcome for the ABS.
The final report of the review was discussed at the Division Heads meeting on 24 August 2004 where recommendations were endorsed.
For further information please contact, Marion McEwin on (02) 6252 7290 or
Sample and Frame Maintenance Through Computer Assisted Telephone Interviewing
The Sample and Frame Maintenance Procedures (SFMP) forms are used to interview ABS business survey respondents over the phone about business structure and ownership changes etc. The paper forms have been converted into Blaise Computer Assisted Telephone Interviewing (CATI) forms so that ABS staff can conduct more efficient and accurate SFMP interviews, create electronic files and easily update other systems. Usability tests were conducted by Forms Consultancy Group (FCG) in developing a SFMP instrument.
The purpose of this test was to simulate proper SFMP interviews using actual Provider Contact Unit (PCU) operators. Twenty-one interviews were conducted. Real data from previously completed paper forms were used for the most common types of interview, so the accuracy of the results could be examined. The outcome was a refinement of the screen design and question wording so that smooth and methodologically sound collection of data could be achieved.
Five PCU operators were used in the tests and other staff acted as pretend respondents. These staff called the operator from elsewhere and the operator asked the SFMP questions on the screen and entered data based on the pretend respondent's comments. This process was observed by FCG staff.
The tests confirmed the effectiveness of our current CATI standards which guided the initial development of the instrument. Some of the major findings include:
- it needs to be very clear what the operator has to do on screens where there is no question, and these screens should be avoided where possible;
- questions where the actual spoken question starts at the top of the screen and finishes with more than one "end" next to each select-button, should be used with care. They don't work if there are notes presented in the middle of what the operator has to say, or if the rephrasing required leaves out key words from the question;
- issues with the structure and words of the SFMP form arose, for example additional "Don't know" options were required for some yes/no questions;
- some formal training in basic use of Blaise CATI instruments is required for all staff using the instruments.
This stage of the SFMP CATI development was very valuable and resulted in a much improved instrument. Further acceptance testing was required when it was integrated into the wider system infrastructure. The CATI has now gone into production.
For more information, please contact, Emma Farrell on (02) 6252 7316 or
Implementation of Generalised Regression (GREG) Estimation for the Retail Business Survey Analysis
In recent years, the ABS has put a major focus on reducing respondent burden. As part of this a conscious effort to better utilise external or administrative data sources has been sought. One of the best forms of administrative data that the ABS can access for business surveys is tax data especially Business Activity Statement (BAS) data. BAS data has been incorporated to reduce respondent burden in the Retail Business Survey through improvement of sample designs via stratification and estimation.
Prior to June 2004 the Retail Business Survey used Ratio Estimation, which utilises one auxiliary variable, Derived Size Benchmark (DSB), to improve the level and accuracy of estimates. DSB is a tax derived item which models the old ABS employment which forms a part of stratification for all ABS business surveys. The retail survey was last redesigned in 1994 on the basis of state, industry and number of employed persons.
Preliminary investigations indicated that gains could be achieved for the retail survey through the use of tax data and Generalised Regression Estimation, but these would be maximised through redesigning the survey at the same time, incorporating the tax data into stratification. MD developed a strategy to coincidentally implement both a change to stratification and estimation.
One problem evident through initial testing of the BAS data was a large number of units with a zero retail sales value but a non-zero benchmark. A large number of these were legitimate due to recent deaths and out of scope units. This problem was limiting the potential gains that could be achieved from the BAS data. A solution to this was to post-stratify in generalised regression estimation by zero and non-zero units to essentially fit separate regression lines to the two types of units. This addition resulted in gains that aligned more with expectation.
A significant reduction in both sampling error and sample size was achieved. This was primarily due to the much better correlation of the BAS data with the variable of interest, retail sales, than the previous benchmark variable. While the improved method of estimation has been significant, the majority of the gains were realised when complementing this with the new stratification.
For more information, please contact, Glenys Bishop (02) 6252 5140
This page first published 22 October 2004, last updated 4 January 2005