15 Scientometric Studies and their Role in Science Policy

Sujit Bhattacharya

I. Objectives

To understand the importance of measurement activity in science for science policy/decision making. To learn the basic concepts of technology forecasting and how scientometric emerges as an important tool for technology forecasting. Highlights thorough examples the application of scientometrics in science policy.

II. Learning Outcome

At the end of this module, you studied measurement activity in science for science policy/decision making. Also, you have learnt basic concepts of technology forecasting and how scientometric emerges as an important tool for technology forecasting. Modules highlighted thorough examples the application of scientometrics in science policy; thus you have also gained knowledge with regard to application of scientometrics to science policy.

III.   Module Structure

1. Introduction

2. What is technology forecasting (TF)?

3. Bibliometric analysis: one of the popular methods in technology forecasting

4. Usefulness in the context of foresight study

5. Example: predicting the growth of scientific field

6. Example: forecasting a Country/company’s probability of success

7. Case studies on application of Scientometrics in Technology forecasting

7.1 Case study of GlaxoSmithKline adopted from Norling et al. (2000).

7.2 Study of the Importance of public funding for research for industrial competency adopted from Broad, W.J. (1997).

7.3 Case study An-Cheng and Chen (2008): Technology forecasting of new materials (e.g. Nano-sized ceramic powders)

7.4 Case Study Chen and Cheng Wang (2010): Predicting the development trend of the walking technique

8.  Summary

9.  References

1. Introduction

The measurement of scientific activity evolved in the late 1960s as a practical need for a rational policy decision in scientific research. The strong demand of an objective method of evaluation of scientific activity came from competitive funding pressure of different streams of research. Other perspectives that supported the objective approach was the advent of more organized scientific research activity/mission mode approach, science playing a much larger role in economy and society, scientific research increasingly requiring larger funding support from government (Edge, 1995). Thus from 1960s onwards evidence based measurement of scientific activities – authentic data collected on scientific activities and analyzed on a regular basis started playing an important role in the formulation of science policy.

Scientometrics is primarily involved in constructing S&T indicators that can properly measure the various aspects of the latent variables of ‘scientific activity’ (for example productivity, quality). Indicators are constructed from the empirically directly measurable variables (for example publications, citations, patents) which are used to judge indirectly the state or dynamics of the corresponding latent variables. Scientometric research has led to the construction of varied types of indicators that can address the following main types of scientific activity:

•      Dynamic aspects – growth of scientific knowledge, signaling the new areas of research, etc.;

•      Structural aspects – mapping the cognitive structure of scientific knowledge;

•      Evaluative aspects- assessment of scientific research; and

•      Prognostic-future studies.

In spite of some major inherent limitations, the approach of analysing scientific activity through S&T indicators has come to stay; the debates has now shifted to how well one can construct better indicators and collect statistics. The scientometrics method has become more popular and relevant because it can reveal the ‘hidden aspects’ of research activity and can signal policy actions. The evidence based studies provide important inputs for policy actions.

One of the influential findings from the analysis of research papers have been the role of research collaboration in the advancement of scientific research (Bhattacharya et al. 2012). The study highlights the role of collaboration in nanotechnology, a fast growing field of research with diverse applications across various sectors. Analysis of research papers in this field show that papers from India in nanotechnology have high impact when the authors belong to different institutions. For example, 26 papers were identified from India in the top 1% cited papers globally in nanotechnology in 2009. Of the 26 cited papers, 16 papers (61%) were found to be collaborative papers. Again examining Indian publication in nanotechnology, journals with high impact factor showed collaboration playing a major role. Looking at high reception of papers from China also brought out this fact. In a sense strong evidence emerges that collaboration has played an important role in pushing the quality of scientific research in nanotechnology for both India and China.

Similarly, examining patenting activity of India and China in nanotechnology in the US Patent Office reveals some interesting trends and insights. The joint patenting among Hon-Hai Precision Industry Ltd. and Tsinghua University is striking. Tsinghua University is a leading University of China, part of C9 League that comprises the top 9 universities in China and is among the top 100 universities in different global university rankings. Hon-Hai Precision Industry is a Taiwan based entity, commonly known by its trade name, Foxconn and is the world’s largest contract electronics manufacturer. The joint patent activity has led to filing of 118 patents during the period 2001-09 accounting for 72% of patents filed by China in nanotechnology. Similarly it has led to the grant of 22 joint patents which is 49% of the patents granted to China during the period 2001-09. The examination of their joint patents provides an indication of the inventive capability that is developing over the years in China.

Further analysis of the trends of these joint patents show the importance of the patents filed for future application development. The early applications are in varied methods of growing carbon nanotubes. Later patents address specific applications of carbon nanotubes i.e. yarn (textile), microscopic electronics, nanoscale integrated circuits, nano based display panels (for computer, LCD, TV and mobile screen), Lithium battery, composite material for automotive, carbon based array sensors and electron emission device. Thus it is observed that the joint activity is not only strengthening over the years but is also directed to specific applications. Important indications like this can be revealed through bibliometric analysis and based on these indications policy/decision making can be strengthened. From the above two examples, the policy makers can observe the important role of collaboration in strengthening scientific and/or technological competency. Suitable institutional mechanisms can be adopted by a country to strengthen collaboration further.

2. What is technology forecasting (TF)?

An important contribution of scientometrics is also found in Technology Forecasting which is primarily understood as all purposeful and systematic attempts to anticipate and understand the potential direction, rate, characteristics, and effects of technological change, especially invention, innovation, adoption, and use. Primarily it refers to a set of methods/techniques to identify future technology developments and their interactions with society and the environment for the purpose of guiding actions designed to produce a more desirable future.

Technology Forecasting (TF) usually focuses on specific technologies, but sometimes the scope is more encompassing. A firm might roadmap a set of related technologies and products; an industry association might roadmap the range of emerging technologies potentially affecting its sector; or a nation could roadmap technologies across its economic base. A new conceptual area/framework has evolved which is called Technology Future Analysis (TFA) which includes the Technology Forecasting as a sub- discipline. This framework articulated by Porter (2004) has found wide adoption. In this context let us know what constitutes TFA.

Technology Future Analysis Primarily Covers:

• Technology monitoring, technology watch, technology alerts (gathering and interpreting information);
• Technical intelligence and competitive intelligence (converting information into usable intelligence);
• Technology forecasting (anticipating the direction and pace of changes);
• Technology Road-mapping (relating anticipated advances in technologies and products to generate plans);
• Technology assessment, and forms of impact assessment, including strategic environmental assessment (anticipating the unintended, indirect, and delayed effects of technological changes); and
• Technology foresight, also national and regional foresight (effecting development strategy, often involving participatory mechanisms).

Among its various applications are: prioritizing R&D, input to public policy, strategic decisions on technology, and so forth.

Some Common Methods of Technology Forecasting1

Expert Opinion: Delphi (iterative survey); Focus Groups [panels, workshops]; Interviews

Modeling and Simulation: life cycle analysis; Systems Simulation; Economic base modeling [input-output analysis]; Technology Assessment; and Scenarios: Scenario-simulation [gaming; interactive scenarios].

Scientometrics and Technology Forecasting

Scientometrics plays an influential role/instrument in the application of the several of the above forecasting methods. For example, in many Delphi study choice of experts is based on identification done through mapping research profile. Authors who are highly cited or research groups/institutes are uncovered which provides an objective and more precise method for expert selection.

On the other hand common forecasting techniques are applied to the bibliometrics [1] data to reveal new insights. For example research publication activity and patenting activity over a period of time is a common approach in trend analysis and technology watch. It provides for the decision maker and or researchers/ research groups to have a more informed understanding of the development in a field and allows for simulation and scenario building (extrapolation through forecasting methods of how the field is expected to develop). This is very useful for decision based analysis such as funding support, roadmap for technology development and impact, undertake strategic decisions, etc.

3. Bibliometric analysis: one of the popular methods in technology forecasting

Bibliometrics is the statistical analysis of text documents, typically publications and patents. Since publications in this case refer mainly to scholarly journals and patents, science and technology intensive industries would logically be a better fit for this type of analysis. As patents and scholarly journals often deal with ideas and techniques relatively in the early stages of technology development, this is the stage at which bibliometric methods for technology forecasting are most useful. In later stages the exploitation of knowledge takes place which requires large amount of tacit knowledge (know how, practical knowledge) which cannot be captured by bibliometrics.

4. Usefulness in the context of foresight study

Help Identify emerging scientific specialties and research areas; research fronts where action is taking place;

Bibliometric data can be adjusted (mapped or compared) using an S curve as a way to fit the technological growth process (applying models such as Fischer-Pry Model); Technology clustering based on: (a) Co-occurrence analysis of patent classification codes/keywords from titles or abstracts; (b) Mapping Patent citation network; Patent citation network can help identify in any area of technology: (a) The technology plane, such as; Precursor and Successor technology; (b) Key patents, key companies, inventors; (c) Identify interactions between science and technology etc.Core competency of countries in an area of scientific research or technology; and Expert selection within a field.

5. Example: predicting the growth of scientific field

The application of exponential growth model to science in terms of the number of units of scientific productivity: growth of scientific papers (Price 1961);

Science like any other growth phenomenon seems subject to a description in terms of logistic growth rather than pure exponential growth;

Why this type of growth happens/ processes or activities behind this Growth?

The existing stock of scientific literature doubles every fifteen years, it does not necessarily mean that this growth rate applies to every single discipline and sub discipline; and

From an analysis of growth data it appears, on contrary that scientific disciplines and sub disciplines may vary widely with respect to growth rate.

The concept of linear growth, exponential growth and logistic growth are highlighted in the figure given below (adopted from Price, 1961). Linear growth implies increase with a fixed quantity of units for each units of time. It is an exponential growth if the increase is in fixed proportion of the total population for each unit of time. Logistic or S shaped growth is a combination of exponential growth and asymptotic growth whereby after an initial exponential phase, growth gradually slows down as the population approaches saturation level.

Adopted from Science Since Babylon (Price 1961)

6. Example: forecasting a Country/company’s probability of success

Patent based Indicators can address to some extent the technology part of the forecasting problem, by analysing: Which of the companies has the most valuable

portfolio of patents? How concentrated is a company’s R&D across different technology categories? etc.

Some of the Indicators that are used in this type of analysis are:

Number of Patents: Granted by a Patent Office (say Indian Patent Office); Number of cited references Per Patent: A count of citations received by a company’s patents (for example year-wise or other-wise);

Current Impact Index: Measure based on how often a company’s patents are cited by other patents;

The number of times a company’s most recent five years of patents are cited in the current year, relative to the entire patent database; and

Technology Strength: The number of patents times the current impact index (will indicate the technology strength of a company’s newly issued patents).

7.  Case studies on application of Scientometrics in Technology forecasting

7.1 Case study of GlaxoSmithKline adopted from Norling et al. (2000).

Just after the merger of SmithKline with Beecham (1990), the question of focus was to guide the company in refocusing its R&D resources. The application of scientometrics or science mapping was used as one element in the redirection of its R&D resources. Scientometric maps of the seven therapeutic areas were generated in which the merged company was also active. From this, they concluded that the field of gastrointestinal disease (GI) research in particular was not generating a significant amount of high- performance research. The company decided to close its research activities in this area, and to focus on research in the remaining six areas. This Scientometric study gave the company an important intelligence perspective that enabled it to reshape its research portfolio for greater productivity, and to define a number of promising technology opportunities.

7.2 Study of the Importance of public funding for research for industrial competency adopted from Broad, W.J. (1997).

CHI Research Study in 2004 demonstrated the importance of public science in generating key input for subsequent patents in technology areas. This study showed the substantial citations to research papers in patents. It also highlighted the role of public funded institutions as almost 73% of research papers cited in patents emerged from public funded institutes (44% from public funded research institutions in America and rest 29% from foreign public funded research institutions). This work contributed significantly to the increase in research grant for public science in the USA. It also got noticed in other countries and had high spin-off effect. i.e. many countries increased outlay for public science.

7.3 Case study An-Cheng and Chen (2008): Technology forecasting of new materials (e.g. Nano-sized ceramic powders)

The authors adopted the bibliometric analysis through EI database and U.S. Patents and Trademark Office (USPTO) database to gain useful data for this work. This study applied the growth curve method to investigate the technology performances of Nano-sized ceramic powders. The study found that Nano-sized ceramic powders were all in the initial growth periods of technological life cycles. The technology performances of Nano-sized ceramic powders through the EI and USPTO databases were similar. The bibliometric analysis was proposed as the simple and efficient tools to link the science and technology activities, and to obtain quantitative and historical data for helping researchers in technology forecasting, especially in rare historical data available fields, such as the new materials fields.

7.4 Case Study Chen and Cheng Wang (2010): Predicting the development trend of the walking technique

The authors extracted patent amount information of the biped robot walking technique in Japan from the patent database of the Japan Patent Office. The study then analyzed the extracted information using the S-curve Loglet Lab, Pearl and Gompertz models in order to predict the development trend of the walking technique in Japan. According to the results of the analysis, the biped robot walking technique in Japan will continue to grow and reach saturation in the period 2079–2082. This type of findings helps business managers and technical developers to manage their strategies of developing the techniques. It is feasible to predict the development trend of a technology using the approach presented in this study.

8. Summary

The unit highlights the utility of scientometric studies in decision making/policy making. Technology forecasting provides an informed view of technology, its development and future trends and hence is an important tool in decision making. The module provides conceptual understanding of technology forecasting and highlights how scientometric methods also apply in technology forecasting.

The module highlights Scientometrics is increasingly being applied in science policy studies. Indicators constructed from scientometrics can reveal the dynamics of a research field, how it is developing, and the core competence of different research groups or nations at large. It can show gaps in a country’s research profile and can identify to what extent its research has the ability to inform technological development (for example examining how often research publications are cited by its patents, etc), or show a firm/country technology preparedness (say examining a firm/country patenting activity in emerging areas such as stem cell research, nanotechnology, etc.).

In technology forecasting, Scientometrics is playing a key role. For example, in many Delphi study choice of experts is based on identification done through mapping research profile. Authors who are highly cited or research groups/institutes are uncovered which provides an objective and more precise method for expert selection. Text mining helps to identify conceptual connections between apparently disjoint areas of research. For example it has been applied in medical research to identify connections between different health problems, etc. Trend analysis based on research publication activity and patenting activity over a period of time is frequently undertaken. It provides for the decision maker and or researchers/research groups to have a more informed understanding of the development in a field and allows for extrapolation through forecasting methods of how the field is expected to develop. This is very useful for decision based analysis such as funding support, create the roadmap, undertake strategic decisions, etc.

Scientometric approach however requires deeper introspection to make it more acceptable to the scientific community and science policy/decision making. Problems of scientometric based research that require attention can largely be categorized under: methodological issues, issues of databases that include data mining and visualization tools, lack of integration of scientometrics research with other disciplines and the need for adopting reflexive/self critical approach. Some key issues are highlighted below.

One pressing issue that calls for attention is the problem of inter-relation between the S&T indicators and latent variables. Are the indicators chosen really measure the latent variable and what aspect of the latent variable it measures requires deeper introspection.

Different databases have small overlap in records. For example it has been estimated that there is only 52% overlap between two popular bibliometric databases namely Scopus and web-of-Science. Non- article items are less covered by standard databases, making the bibliometric approach more difficult. The lack of unification of items in a database is another concern. Mapping and delineation of scientific fields are related issues which call for developing a proper search strategy for extraction of records.

Scientometric analysis reveals various aspects of scientific and technological communication process by exploiting publication, citation and patent data. For example it can show linkages among institutions, concepts, pattern of knowledge growth within sub-grain analysis, identify the key papers/patents, long term research and technology trends, and identify scientific community among others. This rich scientometric data provides ‘object of investigation’ for studies within various disciplines such as sociology of science, science, technology and innovation studies, strategy studies etc. Inspite of this wide reach of scientometric tools and rich data, it has not entered in the mainstream research methods of other disciplines.

9.  References

•  David Edge. (1995) “Reinventing the Wheel,” In Jasanoff et al. (eds.) Handbook of Science and Technology Studies, Sage.
• Bhattacharya, S.; Bhati, M.; Jayanthi, A.P. and Shilpa. (2012) “Knowledge Creation and Innovation in an Emerging Technology: Contemporary and Future Scenario in Nanotechnology”. Available at http://nistads.res.in (under Reports)
• Porter, A.L. (2004) Technology futures analysis: Toward integration of the field and new methods, Technological Forecasting & Social Change, 71: 287 – 303
• Derek J. and de Solla Price (1961). “Science Since Babylon. Yale University Press, New Haven, Conn.
• Norling, P.M.; Herring, J.P.; Rosenkrans, W.A.; Stellpflug, M. and Kaufman, S.B. (2000). Putting Competitive Technology Intelligence to Work, Research-Technology Management, 43(5): 23-28.
• Broad, W.J. (1997). Study Finds Public Science is Pillar of Industry. New York Times, May 13.
• Cheng An-Chin and Chen Chia-Yon. (2008) The technology forecasting of new materials: the example of nanosized ceramic powders. Romanian Journal of Economic Forecasting, 5(4): 88-110.
• Chen-Yuan Liu and Jhen-Cheng Wang. (2010) Forecasting the development of the biped robot walking technique in Japan through S-curve model analysis, Scientometrics, 82:21–36. DOI 10.1007/s11192-009-0055-5.