The use of the survey instrument in a Legislative Budget Office context
18 April 2017, by Usman W. Chohan

It has been previously noted that there are inherent challenges in measuring the effectiveness of Legislative Budget Offices (LBO). Although it is clear that they do provide “some form of good” for the budget process, through their role in enhancing budget transparency and fiscal accountability, as well as their assistance to legislatures in budget oversight, all of this is difficult to measure or substantiate in a concrete manner.

A previous article in the Austaxpolicy blog proposed the use of survey instruments to solicit feedback from important stakeholders, such as legislators in both government and the opposition. Feedback from parliamentary political parties on the LBO’s work would, in theory, provide an excellent manner of assessing its usefulness. In particular, it would provide insights into areas of improvement required by the institution.

However, there are limitations on the use of survey instruments in an LBO context, as a recent discussion paper has identified.

First, survey instruments are an inherently subjective measure. This is important to note because “doing a good job” will be very much a function of the expectations that legislators hold. Some legislators may set low expectations and be content with a passive “costings role”. Others may have higher expectations, particularly when they are in the opposition, expecting a more “activist” budget office that follows an open-publishing approach with strong ties to the media.

Second, a survey instrument can be vulnerable to politicization, as some legislators may use the survey instrument to provide negative feedback as a form of reprisal. As a result, the survey instrument may not be an adequate reflection of the work that an LBO does. This is because the work of an LBO challenges the legislators in the fiscal process to a certain degree.

Third, the survey may turn out to be an extremely costly enterprise, as the case of Australia’s PBO stakeholder survey shows. In 2015, the PBO had a survey instrument sent out to more than 3,000 stakeholders, but only got 63 responses in return. Worse yet for the taxpayer, the PBO sought an external consulting firm to disseminate the survey, which brought a pricetag of roughly $23,000. In a Senate Estimates hearing, ACT Labor Senator Katy Gallagher expressed her amazement at the exorbitant cost of sending a survey to so many people, saying that “It’s money for jam, really: 63 people responding for $23,000 dollars?” In response, Parliamentary Budget Officer Phil Bowen admitted that, “[in] hindsight we sent that to a lot of staff who we probably should never have.”

Read the full article at Austaxpolicy blog.