Measurement Issues in A2K Panel
From A2K Wiki
Speakers: Bruno Laporte, Karine Barzilai-Nahon, Amitabh Kundu, Ernest Wilson Moderator: John Horrigan
Contents |
Bruno Laporte
Access of knowledge is very difficult to define. Even if you think people can have access to knowledge, this is only the first step. It is important to understand about the environment in which countries can tap into the global knowledge platform and use the knowledge for their needs. The ultimate goal is the improvement of like of people.
Several indices have been created: Some talk about knowledge in general terms, some are more specific. There have been several attempts to measure, but the problem is that many tend to focus on only one dimension, none really have a heuristic approach to the whole access to knowledge paradigm. This, of course, is related to the difficulty in defining access to knowledge. There is also a problem of the purpose of measuring -- need to know how you're going to use it.
Definition of Access to Knowledge Economy - uses knowledge as the key engine of growth. This is an economy where knolwedge is acquired, created, disseminated, and used effectively to enhance economic development. At the World Bank, we ar ethinking about transition
The Knowledge Assessment Methodology is the result. This provides a basic assessment of Knowledge Economy readiness and gives relatives strengths and weaknesses as a way of identifying appropriate policies and investments for transition to the Knowledge Economy. See www.worldbank.org/wbi/kam for more details.
Overview of the KAM: 128 countries. The comparison is made across 80 structural and qualitative variables. Data are updated semi-annually; country coverage expanded whenever possible. We have a basic scorecard based on 12 key indicators and cross-country comparisons and ranking. This allows an easy graphical comparison between countries. Difference between 1995 and today are helpful for looking at changes over time. In addition, it can be provided at a federal or state level for countries for which it is relevant.
The KAM is a transaprent tool that is constructed from data published from reputable sources. Ordinal normalization and ranking is straightforward, and easy to explain. It is user friendly and allows to select variables that are most relevant.
The KAM is a tool for policy makers. it provides a relatiy check on performance relative to outher countries. In addition, it identifies the most urgent interventions, such as innovation systems in India and life-long learning in China. We have been able to bring people with very different backgrounds together to engage in a dialogue.
Amitabh Kundu
The level of adoption of ICT and the impact of ICT has been quite well analyzed for highly developed countries. Unfortuatne; not available data, changes over very short periods of time, and most importantly there is a huge diversity between countries so it is hard to determine the impact on the knowledge system. However, this is incredibly important. Quantification increases the capability to demand changes and/or help make these changes.
Over the past decade, the database has improved significantly. Mathematical models have also started being used by countries to look at the countries. But most of the models give emphasis to the supply side of ICT while not looking at demand from the users. This is related to the fact that these models are borrowed from the developed countries.
What are the problems that current in the context of these models: 1) These models look at short term gains and investment; 2) Digital divide is very clear: there is high inequality to begin with in these countries and these technology makes it more; 3) Regional inequalities - only developed regions take advantage of the technology; 4) Gender divide - women do not have access in less developed countries; 5) ICT goes into area that has higher rates of return rather than social issues, medical issues, etc.
We divided the countries of the world into 4 groups. We ran regressions at two time points and looked at ICT indicators and development indicators. Level of ICT adaptation, cost and efficiency, social sectors, socially marginalized people's benefits. We had to use proxies for some countries, so there are some data gaps. We then ranked the 9 Asian countries according to this study.
One thing that has struck me is weighting different variables in an ad-hoc manner. This has been addressed in welfare economics, where you have individuals ranking different alternatives. They find that it's not possible to compose those into one group. Therefore, we propose an axiomatic framework. That permits a ranking.
It is also important to not lose information - when we move from numbers to ordinals and back, we lose information. Instead of doing this, we divided each value by its mean to keep the standard deviation.
Karine Barzilai-Nahon
I will concentrate on 3 arguments: 1) Policymakesrs prefer to approach A2K issues from a hard-data and cost-effective standpoint; 2) ICT and networks are not neutral bodies not from a social standpoint, but as a structure - I think structurally they are very political in nature; 3) I reccomend building a more holistic and comprehensive measure rather than measures on individual topics.
I don't think that the issue is statistical challenges - I think it is a conceptual issue. It's easier to look at data from institutions that exist already rather than going to the source. People think this is cost-effective. But it is only cost-effective for now, not for later. The goal of the policymaker is to compare. Comparisons are not objective. Can comparative and contextual weave together? Does comparison have to be objective? No. We can construct a measure that is both comparative and contextual.
About the level of observation, most of what I have observed has been on the international level. Policymakers like to know where they are in reference to other countries. Ignoring smaller units within society will not be cost-effective over the long run. If we ignore certain communities because we can't measure them, then we lose information that we shouldn't lose.
As to method of approaching the data, most measurements are variables that look at only one aspect of access to knowledge. I don't mean we lose out on factors, but we have something at the end that's an arbitrary index where people arbitrarily decide how important each issue is. I suggest that we look more holistically. We need to look at connections. I'm not saying the index should be only social and political, but that we need to look at the context.
Ernest Wilson
Let's put ourselves in the position not only of scholars, but rather as consumers of knowledge. We need to look at the data that scholars produce. What I want to argue is that as scholars we need to think about the purpose of the analysis, which will tell us what kind of data we need to try to get from the world. How we do this depends on what we want. For example, the KAM approach is very helpful when you're dealing with senior level analysts.
I want to report on a project that starts at the bottom and works up, but I think there is a fit between that and the way of starting at the top. To do this, we need to think in a multidisciplinary way.
Some measurements that have been used are e-readiness, GDP, IT, enabling environment, and others. I want to point out some limitations to these approaches. 1) they tend to be static; 2) they are often very aggregated which leads to problems; 3) they tend to pay attention mainly to technology, societal factors are an appendage; 4) they are often less concerned with equity than they should be and tend to be apolitical; 5) they often work on a poor knowledge base. Ee would want a more dynamic and historical model that is embedded within society and deals with distributional issues.
What steps can be taken to encourage individuals to access the information that is available to them. How do we measure differential levels of empowerment?
What people need is stories about successful instances of how access to knowledge has been used by people to make them want to access the information. We try to get a more frank, more individualized conversation going. What we did is first realize that these resources are scarce. We tried to identify a set of critical negotiation issues over which people contend. These included privatization, commercialization, access to networks, access to facilities, VoIP, etc. We didn't just articulate stories but instead looked at stories in a context of particular points of contention. We then were able to compare these between nations. In combining the quantitative with the qualitative, we were able to tell stories that were helpful to our audience.
