As outlined in the section on Transparency and the right to information, the concept of transparency is often very difficult to define and to accurately measure and just as there is no commonly agreed definition of "transparency", there is no consensus on how it should be measured.
But since transparency is seen as a mechanism for promoting accountability, then one key way of measuring transparency is to measure the amount (and scope, accessibility, quality, reliability, accuracy and timeliness) of information disclosed and/or made publicly available.
Global Witness, one of the five partner NGOs working on Making the Forest Sector Transparent, has, for example, previously used quantity, quality and credibility of information as a measurement of transparency in its Independent Forest Monitoring
To take this measurement process one step further, the partners of this project, through a participatory process, have developed a transparency report card as an assessment tool.
A report card is basically an assessment tool. Whilst report cards are popular in other sectors, the use of a transparency report card for the forest sector is innovative.
Broadly speaking, there are two main models: (i) Scoring and Ranking or (ii) Descriptive.
Scoring and ranking report cards are quantitative in nature. Scores are awarded according to a standardised scale (anything from simple binary scoring -- 1 or 0, for example -- to different categories from which to choose, such as green yellow or red.) The scores obtained are usually aggregated into a single index. Since any scoring implies some sort of weighting, rankings can be established and broad comparisons can be made between units of analysis.(Image shows an example of ranking from Open Budget Initiative).
Descriptive report cards are more qualitative in nature. They rely on the compilation and analysis of a series of different criteria and indicators. Descriptive report cards allow for a greater level of analysis on a case-by-case basis.
Some of the key advantages of report cards as assessment tools include:
By way of example, the report card developed for Making the Forest Sector Transparent gathers data by asking 70 questions split across 15 agreed-upon "themes" -- each of which is a measure of accountability. Data is collected in each country on a yearly basis.
Under each of these general themes, there is a sub-set of related questions: for example, an overall question might be "Is the permit allocation process transparent?" and respondents can answer "yes" or "no" to this question. But the answers to a series of more probing sub-questions helps us to understand how they have arrived at this yes or no answer and help guide any advocacy work. For example, respondent are then asked to clarify "Do permits exist for all users/services?" and "Is it clear who decides if/when to allocate permits?" and so on.
Report cards are only useful as an assessment tool if you very clear about what it is you want to assess and if the information you collect is relevant and reliable. It is therefore crucial that, during the design stage, you: