A research-innovation model to foster university-industry links: an empirical typology

Lokman Zaibet^a* , Tarig Gibreel, Sheikha Akhzami, Rachid Hedjam, Houcine Boughanmi

Sultan Qaboos University, Oman. Corresponding author's email address: lzaibet@squ.edu.om

Keywords: Innovation process, University, Agriculture, Food, Oman

ARTICLE HISTORY: Received:02-Jul-2019, Accepted: 11-Oct-2019, Online available:01-Nov-2019

Contribution/ Originality

This paper contributes to the existing literature by empirically construct a model of research-innovation that would improve the impacts of research and build the missing links between the University and end users. It introduces logical phases in the innovation process based on the innovation dimensions namely the technical, policy and social dimensions.

1. INTRODUCTION

The relation between innovation and productivity/economic development is well established (Jorgenson, 2011; Chang et al., 2015). It is noted that (innovation) patents enhance productivity growth, and this relationship is stronger in countries where patents are widely held by small private firms (Chang et al., 2015). Innovation, in all dimensions, stands as the main driver that impacts food systems and their ability to deliver nutritious and sustainable food products. Innovation has been a significant engine for food system transformation and is essential to address the needs of a rapidly growing population in the context of climate change and natural resource scarcity (FAO-HLPE, 2017).

Recent research on farmers' innovation has focused on the relationship between farmers' ability to innovate and participate in commercial networks (Boahene et al., 1999) and tactical alliances (Hagedoorn and Duysters, 2002). These findings apply to transfer of technology models where research users (industry) and innovation providers (scientists) are co-creators of knowledge and innovation (participatory innovation approach), and where many actors are involved in the process of innovation, considering institutional and political constraints (innovation systems).

There is also growing literature about the critical role of universities in generating innovative activities in the public sector and public organizations (Demircioglu and Audretsch, 2019). Publicly and privately funded universities, pressured by the granting agencies, are increasingly focused on research that leads to factual applications and tangible impacts. The trend in research-innovation involves intellectual property disclosure to technology transfer institutions and related industries to boost productivity and overall economic growth. This has been followed by a clear shift from the output-based research paradigm to outcome - innovative performance (Hagedoorn and Cloodt, 2003; Weiss, 2007; Cathy et al., 2010). The relationship between the university and the industry (end users) is revisited by emphasizing the impacts of research while considering the "open-innovation perspective" and "overcoming open-innovation challenges", which requires identifying customer needs and scanning for future disruption (Jonathan et al., 2018).

The research-innovation process would be enhanced if universities changed the way research is done, outcomes are protected, and scientists are geared toward these paradigm imperatives and future developments. Findings show a gap between outputs and outcomes, called the efficacy-effectiveness gap (Weiss, 2007). Universities are addressing these pitfalls following recent research on technology transfer and policy by placing stronger emphasis on academic engagement and commercialization of knowledge (Perkmann et al., 2013; Huang-Saad et al., 2017). Closing the outcome-impact gap also requires dialogue between scientists and economists to unify knowledge and develop new tools accountable to society and better able to set research priorities (Antle and Wagenet, 1995).

There is extensive literature about university-industry links and the process and impacts of innovation. Most publications focus on organizational issues or the phases of innovation management. Perkmann and Walsh (2007) emphasized search and match processes between universities and firms, and the organization and management of collaborative relationships. Organizational arrangements between firms and universities range from research alliances (Hagedoorn and Duysters, 2002) to innovation-centered collaboration along the supply chain (Harabi, 1998). Perkmann et al. (2013) offer a review of the literature on university-industry relations to which they refer as 'academic engagement'. Their approach is based on the synthesis of published research and comparison of obtained results to the involvement of academics in commercialization.

The Sultan Qaboos University (SQU) in Oman has a highly recognized record of research in the fields of resource management, agriculture production, and food science & technology. These research outputs are expected to result in innovations that help to transform agriculture and food systems. The 'expected' impacts are neither well documented nor measured. Following the outcome-impact paradigm, this paper endeavours to understand these gaps and rebuild the research-innovation process using empirical typology. The paper uses a sample of research projects in the fields of agriculture and food to i) understand (and construct) the current research-innovation process, based on ii) a first-hand assessment of the supply perceived impacts of research production. Our typology started by assessing the perception and performance of university scientists with regard to the outcome-impact model. We used clustering techniques to obtain a model of research-innovation development. The model displays various stages of research-innovation where "engagement, advocacy, and achievement" are in line with the different dimensions of innovation; in particular the technical, policy, and social dimensions.

1.1. The research-impact pathways: the process and dimensions of innovation

The trend in research pathways is to emphasize impacts of research, i.e. to "report to taxpayers...on the societal value produced by their money entrusted to (research)" (Weiss, 2007). Research-innovation models are typically based on the research log frame: input, activities, outputs, outcomes, and impacts. Resources are mobilized and employed to bring about the intended program changes or results (Kellogg, 2004). While intended results include outputs, outcomes, and impacts, a distinction is made between outcomes and impacts; the latter is defined as the long-term resulting benefits to end users (Cathy et al., 2010).

Measurements of the outcome-impacts of a project involve the development of a "logical-framework", which is a series of if-then statements describing the path from engaged inputs/activities to outputs and the desired outcomes. In research the process involves the following components: inputs are initially resources put into the process; then research (realized activities and results) is published in journals (output). Journal articles are read by practitioners and decision makers (primary outcome). These users implement a change in practice (intermediate outcome); and a practice change leads to improvement in the target population (long-term outcome) [see United Way, 1996]. In impact assessment studies this approach is applied ex-ante and ex-post and requires identification of outcome-impact indicators at the design stage of the project. Impact indicators are the milestones along the way to help guide actions, document progress, and take corrective actions; they reflect the smaller scale contribution of the project underway as progress is made (Nyangaga, et al., 2006).

The outcome mapping framework has been promoted to (international) research institutions (Nyangaga et al., 2006) for its emphasis on actors' behaviour in the research-innovation system. The framework was retrospectively implemented to describe the intentions of the projects and their outcomes. Such framework identifies and describes the post-research actions that played essential roles in the achieved innovations. University research impact is traditionally assessed using the citation analysis method and performed by examining an individual publication and assessing how often it has been cited, if ever, by subsequent publications (Nicolaisen, 2007). Critics to these models are now recommending "moving from outputs to outcomes" (Weiss, 2007) and assessment of research impact "beyond citation analysis" (Cathy et al., 2010).

The outcome-impact framework is particularly important in the changing paradigm of university-industry relations; in the pursuit of public funding with stringent conditions the university is concerned about economic and social impacts. The university produces knowledge, new ideas, discoveries, and inventions, which are ingredients of innovation; impacts are sought by business incubation and technology commercialization to boost adoption of innovation. On the other hand in pursuit of competition and competitiveness the industry engages in "open innovation" rather than relying on internal R&D (Chesbrough, 2006). The industry collaborates with universities as knowledge generators to maintain their market competitiveness. Universities are expected to play an open role covered by the term open innovation to develop and commercialize niche technologies through university-industry links (Becker and Eube, 2018). In the face of intensive global competition university-industry collaboration has been advocated by the government as a form of open innovation to enhance the development and commercialization of niche technologies for the environment (Lam et al., 2012).

Jonathan et al. (2018) identified several problems facing organizations to engage in open innovation: attitudinal barriers to networking, the attraction of external experts to develop networks, integration and processing of information, and formulation of effective responses to intellectual property challenges. According to Zaraychenko et al. (2016) the framework of open innovation requires networking at various levels in order to effectively develop new technologies and utilize university-industry capabilities and resources. This framework needs to develop a model of multi-interaction in the innovation process, which ensures the acceleration of transforming innovative ideas into products.

The University-Industry research-innovation model is also concerned with other dimensions and challenges of innovation. The dimensions in question include social, economic, policy, and institutional aspects which stem from the nature of innovation, and also the characteristics of the "current society model in its economic, political, and social dimensions related to economic growth" (Erica et al., 2016). Research shows that university-industry links are described as "network relationships" rather than "arm-length" or "transactional market links" (Chesbrough, 2006). The innovation network is not operating in a vacuum; it requires an emphasis on institutional setup, industry-university trust, flexibility in regulations, efficient communication, and other institutional arrangements (North, 1991).

In sum, to enhance the research-innovation impacts we need an accurate understanding of the nature and sources of innovation, and the mechanism of change or innovation process. Universities and other research institutions are the primary sources of innovations. They are producing knowledge, new ideas, discoveries, and inventions, which will transform into innovations. The industry-university-research models are classified into four types (Shuilong, 2010): (i) government leading mode, (ii) enterprise leading mode, (iii) universities or research institutes leading mode, and (iv) common-leading mode. University-industry collaboration is now framed in an 'open innovation' paradigm which requires new rules of social, economic, and policy aspects, in addition to an adequate institutional setup. The mechanism of economic change, with reference to the original work of Schumpeter, depends on: (i) The causative factor of change: the innovator, (ii) The factor of change: the innovation; and (iii) The interaction of the innovator with the forces at work.

The degree of adoption of research outputs is associated to miss linkages that need to be re-established in order to enhance research-innovation impacts. In particular, this is fixed by setting up a process of research - innovation for the research deliverables. Researchers as well as industry and government executives complain that research findings and technological innovations in the field of agriculture and food are faced with some resistance on the part of the communities intended to benefit from these new ideas and technologies. The literature is rife with examples explaining this resistance by recourse to predetermined ethnocentric conclusions and value-judgments such as ignorance or lack of knowledge. Alternatively, we propose an approach based on the need to understand the innovation process and related dimensions.

2. METHODOLOGY

In this paper, we use the research to impact pathways log frame. This approach is applied in different fields such as in agricultural research (Lynam et al., 2009) and medical research (Weiss, 2007). This framework emphasizes impacts of research and "provides the rationale for investment, the organizational framework for the research, the character of the network of partnerships and the motivation for staff" (Lynam et al., 2009). A questionnaire was designed based on the research-innovation log frame as outlined in the previous section, covering the following areas:

1. The identification of the project scientist(s) and collaborators,
2. The type of grant and fund provider,
3. The project ideation, sources, and tacit knowledge,
4. The planned objectives and "perceived" outputs, outcomes and impacts,
5. The assessment of the efficacy of research-innovative performance (based on scientists perceived indicators),
6. The 'conditions' required to build the missing links and to improve overall impacts.

These areas cover the different phases of a research process, from ideation, funding, setting objectives to outcomes and impacts. They also consider areas of improvement which capture the enabling conditions (dimensions) of the process. The data will enable to (i) assess realized innovation efficacy, (ii) construct the current process, and (iii) propose a research-innovation model with required links.

2.1. Sampling and sample characteristics

We applied the above framework to a sample composed of 24 completed projects selected from all projects of the College of Agricultural and Marine Sciences (CAMS), carried out from 2007 to 2014. The college of Agriculture is considered to be the most highly active college in the university in terms of grants awarded and publication rates. The sample represents 16% of total CAMS projects since 2007 and covered 40% of the current faculty members in the college. Only research projects with an existing faculty are selected to carry out the surveys. The projects differ in nature and in scope. To cope with heterogeneity issues in projects, a stratified sampling based on fields and grant type to capture different research and innovation results (Georghiou, 1999). Soil Water Agricultural Engineering, Natural Resource Economics, Marine Sciences was grouped to capture policy-driven projects, Animal and Veterinary Sciences and Crop Sciences were grouped under Farm oriented projects, and Food Science and Nutrition capture industry-driven innovation.

The sample shows a prevalence of internal grants (52%) followed by strategic project funding and consultancies (25% and 21% respectively). The sample is not randomly selected but instead purposely designed to include all these categories. The research teams are composed mostly of scientists (63%) against only 4% of teams with field practitioners and policymakers, and 33% of teams with scientists and field practitioners. Project idea (ideation) is a balance between previous experience, original and demand-driven. There is a clear correlation between the project idea and the source of funds, where the demand-driven fund is mostly coming from external strategy and consultancy sources.

2.1.1. Clustering method

Data collected over 24 faculties/completed projects consisted of measures on five points Likert scale of the different variables as outlined above. The data provide i) an understanding of the research process and model and ii) a first-hand assessment of research impacts as perceived by university scientists. Based on these first results, we then constructed a typology using clustering method to understand the features of different categories of researchers concerning innovation, networking, and linking to industry and overall community users.

Feature clustering proposed in the literature includes mixture models, normalized cuts, finite mixtures, feature similarity, neuro-fuzzy, hierarchical, Markova clustering approaches (Hedjam and Cheriet, 2012; Ienco and Meo, 2008). The proposed method includes two main phases, feature clustering, and individuals' networking. The first phase consists in separating the features into a small number of homogeneous clusters, while the second phase consists of computing the degree of similarity between different individuals and build the corresponding connectivity network. Individual networking can be modelled as a non-oriented graph. Each node represents a unique individual, and each edge between two nodes represents the similarity between the two corresponding individuals. The graph is built from the so-called Affinity (Similarity) Matrix. The latter is, in fact, a square matrix M of n rows and n columns, where n is the number of individuals, and each entry M(i,j) holds the similarity measure between two individuals. Formally, let define each individual by one feature vector F of d features, i.e., F=(f_1,f_2,..,f_d ). The Euclidean distance between the i^th and j^th individual is defined as follow:

And the corresponding similarity measure is defined as follow:

Where s is an adjustable parameter (in this work s is set to 0.1). From Eq. (2), the similarity M varies inversely proportional to the distance E. In other words, the smaller the distance between two individuals is, the higher the similarity between them.

3. RESULTS AND DISCUSSION

3.1. Efficacy of research: outcome-impact criteria and assessment

As indicated in the methodology, we aimed to understand the perception of project scientists in the research and impact process. Interviewers indicated the following objectives (beyond publication, which is a distinct output): capacity building (37%), capacity and process development (28%), community outreach (14%), growth and welfare (14%) and commercialization (5.7%) Interviewers were asked to indicate which criteria they consider important to assess the efficacy of their research and based on these an assessment is made on a five-point Likert scale (1 is weak, 5 is high). Figure 1 shows results based on the stated objectives (above) and resists when we introduced the concept of outcomes (below). The highest criterion according to these results is publication (scientific community), followed by capacity building. All others are rated very low.

Figure 1: Research efficacy criteria and level (5-points Likert scale): Objectives (above) and outcomes (below)

By introducing the term impact of research there was slight change in answers and key criteria are as follows: capacity building, policy adoption, technology transfer, (benefits to) scientific community, benefits to donors, and (benefits to) community. Overall assessment (average) of impacts is very low (below 1.5). These values, however, differ from one field to the other; Natural resource economics, for example, indicated the highest impact for policy adoption, food science indicated scientific community and product development. Although most grants are public sector-oriented (internal grants and strategic funds) benefits show that the least targeted are farmers and industry; public sector comes 3rd. The first target overall as students and the scientific community.

Figure 2: Impact assessment of research: College (above) and department of NREC (below) Clusters' analysis and innovation process

In this study, we used a hierarchical clustering-based method for the investigation of the relationship between features in the dataset on hand. Specifically, we opted for Ward's hierarchical clustering method due to its simplicity and efficiency. Ward's hierarchical clustering is one of the most used methods in machine learning. It is a 'bottom-to-up' agglomerative hierarchical procedure, which seeks to build a hierarchy of clusters. The strategy is to iteratively, merge clusters into larger clusters. An objective function determines the best candidate clusters that will be merged in a new cluster at each iteration (Ienco and Meo, 2008). A simple minimum variance criterion, which minimizes the total within-cluster variance, can be used as an objective function of the algorithm.

The computed Similarity Matrix is then used to produce the corresponding graph shown in Figure 3. The letters indicate abbreviation of individual names in the sample. The networks show three distinct groups with different sizes, and dispersed individuals not linked to any sub-group. Next, we try to study these clusters and their distinct features.

Figure 3: Individual network (most similar individuals are linked)

Based on Figure 3, we display the features of the different clusters in Table 1. The full list of variables in each cluster is displayed in appendix (1). Shortlisted features (3 to 4 variables) as presented in Table 1 are decided by setting the cutting point at the variance plus the standard deviation (see appendix).

• Cluster 1 is characterized by the objectives included in the proposal (capacity building), the type of innovation sought (know-how) and target population (students). These features indicate a cluster of education-oriented research and as such stands in the starting phase of research-innovation.
• Cluster 2 is one characterized by technology innovation (new product development), considered tacit knowledge in proposal design (social sciences) and ended up with a success story. These researchers are considering the social dimension of innovation and could be categorized as achieving in the innovation impact model.
• Cluster 3 is characterized by research outcomes (capacity building), grant type (consultancy vs. internal or strategic grants) and benefits to target users (Ministry). This cluster is labeled engaging, considering the technical dimension of innovation.
• Cluster 4 is characterized by the objectives included in the proposal (community outreach) and related conditions to improve impacts such as policy making and implementation, and platform development. We labeled this cluster as advocating. At this stage, innovation considers the policy dimension.

These featured characteristics are then interpreted in terms of dimensions of innovation and are indicative of the innovation process phases from the output, outcome to impact (Figure 4).

Figure 4: Innovation phases and dimensions

Put in order, the clusters presented in Figure 4 are indicating researchers' standing in the innovation pathways as: 1) Starting, 2) Engaging, 3) Advocating, and 4) Achieving. These features describe the innovation process. There is a link between these phases and the related innovation dimension considered by the researchers from the research proposal design to research outputs and outcomes.

At an early stage, researchers are more inclined to capacity building (mostly student-oriented) with a focus on generating knowledge and know-how. As discussed earlier, these activities are outputs of academic activities. The population in this category is on the track of research-innovation pathway but only starting the journey. Outcomes and impacts are not well-thought in the first place.

Next, we find academics that are engaging in technology development for farmers' communities. From the design of research projects, these researchers seek collaboration with the Ministry of agriculture, with internal grants (mostly) or strategic funds by the government. Academic engagement is defined as the collaboration between academic researchers and users, which include activities such as collaborative research, contract research, and consultancy; this is different from commercialization, which means activities involving patenting and licensing of inventions (Perkmann et al., 2013). Researchers in this category are mostly dealing with Ministry (of agriculture) to find technical solutions, in addition to capacity building (activities targeting Ministry technicians besides university students).

At a third stage stand researchers who are advocating policy-making, regulations, setting up of innovation platforms to improve research impact. These researchers are raising concerns and conditions to improve the innovation process. Promoting research using advocacy is not straightforward and requires in itself a model of advocacy. Advocacy and engagement are interconnected. Kellett (2009) defines advocacy as "a particular mode of engagement or reflexive academic practice". Besides engaging in "technical" solutions, advocating academics are also aware of the policy dimension and institutional requirements of achieving impacts.

Researchers who are achieving - as shown by a success story - designed their research by considering the social dimension and seeking tacit knowledge from social sciences (economics and sociology). This is where a tangible impact is realized and could be measured and therefore is considered in the Figure as the end of the research-innovation pathway.

The target population in the two cases (achieving and advocating) is different; in the first case the target is the industry (new product/process development), and in the second the target is farmers' communities. In both cases, however, and in open innovation, key to research impact and success is the due consideration of the social and policy dimensions. While governments are advocating university-industry collaboration as a form of open innovation to enhance the development and commercialization of niche technologies, researchers as well are advocating government involvement in setting and enforcing regulations that foster the links between the university and the industry/community users.

Achieving (impacts) is the target of research done by universities in an open innovation approach. Impacts are required by funding institutions but are also conditions for further funding and collaboration with innovation users. The model outlined above shows that innovation and impacts should be included in the design of the project. While starting academicians rely on internal grants, other sources of funding should be sought, including consultancy and strategic projects. Besides funding, innovation and impacts entail policy and social considerations. These conditions are met with more collaboration among researchers in different fields.

Table 1: Hierarchical clustering: variables and dimensions

4. CONCLUSIONS AND IMPLICATIONS

Impacts are confined to publications (scientific community target) and capacity building (student target). The impacts defined by the study are shallow. These perceived influences are the result of the current funding system predominately based on internal grants. This is also the result of the design of research projects from ideation and collaboration to end users of engagement. The subsequent model after clustering analysis displays different stages of research-innovation, showing engagement, advocacy, and achievement as distinguishing features. These features are consistent with the different dimensions of innovation; in particular the technical, policy, and social dimensions.

The current research-innovation model at the Sultan Qaboos University is an output-based model; outcomes of the objectives of the survey were to find the missing links in the output-outcome-impact model. Such links are deemed to improve the performance of the overall innovation process and system. According to this study there are conditions, as stated by interviewers, to improve the innovative performance of university research: funding and international collaboration, research-industry/policy making platforms, infrastructure, and institutions/regulations. The phases of innovative performance would come from changes all along the value chain: changes in the nature of input, activities, output, outcomes, and impact indicators.

The research-innovation ecosystem is constructed by mapping the current innovation system and understanding of the links as designed by our methodology and investigation. Based on the (missing) links, conditions (requisites), and actual research-innovation mapping, figure 5 shows a revised map displaying actors, links, and roles which would improve the current system.

The improved system would capitalize on the generated knowledge focusing on entrepreneurial activities such as incubations, spinoffs, and science parks. This requires the university to establish new units to facilitate this shift in its roles called Technology Transfer Offices (TTOs). The TTO at the Sultan Qaboos University is the "Innovation and Technology Transfer Centre ITTC" which was established in January 2010 as a department and elevated to a center handling the invention protection, commercialization, technology services, and awareness activities. The establishment of this department was a crucial step accompanied by an increase in research funding, research volume in publications, and robust research infrastructure, especially in applied colleges. The ITTC should complement the university academic programs by nurturing talent to foster entrepreneurial activities and manage the intellectual property assets of the university by providing intellectual property protection, entrepreneurial training, active industry linkages, preincubation support, and to be the national model for innovation and entrepreneurship system from academia to industry for institutions of higher education.

Figure 5: A platform for research innovation

References

Antle, J., & Wagenet, R. J. (1995). Why scientists should talk to economists. Agronomy Journal, 6(87), 1033-1040. doi.org/10.2134/agronj1995.00021962008700060001x.

Becker, B. A., & Eube, C. (2018). Open innovation concept: integrating universities and business in digital age. Journal of Open innovation: Technology, Market, and Complexity, 4(12), 91-96. On line https://doi.org/10.1186/s40852-018-0091-6.

Boahene, K., Snijders, T., & Folmer, H. (1999). An integrated socioeconomic analysis of innovation adoption: the case of hybrid cocoa in Ghana. Journal of Policy Modeling, 21(2), 167-184.

Cathy, C. S., Ellen, K. D., & Holmes, K. L. (2010). Beyond citation analysis: a model for assessment of research impact. Open Access Publications, 98(1), 17-23.

Chang, X. R., McLean, D., Zhang, B., & Zhang, W. (2015). Innovation and productivity growth: evidence from global patents. Available at https://ssrn.com/abstract=2371600 or http://dx.doi.org/10.2139/ssrn.2371600.

Chesbrough, H. W. (2006). Open innovation: A new paradigm for understanding industrial innovation. In Chesbrough, H. W., Vanhaverbeke, W., & West, J. (Eds.), Open innovation: Researching a new paradigm. Oxford: Oxford University Press.

Lam, J. C. K., Hills, P., & Ng, C. K. (2012). Open innovation: A study of industry-university collaboration in environmental R&D in Hong Kong. International Journal of Technology, Knowledge and Society, 8, 83-102. doi.org/10.18848/1832-3669/cgp/v08i06/55673.

Demircioglu, M. A., & Audretsch, D. B. (2019). Public sector innovation: the effect of universities. The Journal of Technology Transfer, 44(2), 596-614. doi.org/10.17161/merrill.2014.7810.

Erica, S., Oliveira, V. D., & Gomez, C. (2016). Dimensions of social innovation and roles of organizational actor: the proposition of a framework. Mackenzie Management Review, 17(6), 102-133.

FAO-HLPE. (2017). Nutrition and food systems. A report by the high level panel of experts on food security and nutrition of the committee on world food security, Rome. http://www.fao.org/fileadmin/user_upload/hlpe/hlpe_documents/HLPE_Reports/HLPE-Report-12_EN.pdf.

Georghiou, L. (1999). Socio-economic effects of collaborative R&D-European experiences. Journal of Technology Transfer, 1(24), 69-79.

Hagedoorn, J., & Duysters, G. (2002). External sources of innovative capabilities: the preferences for strategic alliances or mergers and acquisitions. Journal of Management Studies, 39(2), 167-188.

Hagedoorn, J., & Cloodt, M. (2003). Measuring innovative performance: is there an advantage in using multiple indicators. Research Policy, 32, 1365-1379.

Harabi, N. (1998). Innovation through vertical relations between firms, suppliers and consumers: A study of German firms. Industry and Innovations, 5(2), 157-181.

Hedjam, R., & Cheriet, M. (2012). Hyperspectral band selection based on graph clustering. 11th International Conference on Information Science, Signal Processing and their Applications (ISSPA). IEEE.

Huang-Saad, A., Fay, J., & Sheridan, L. (2017). Closing the divide: accelerating technology commercialization by catalyzing the university entrepreneurial ecosystem with I-corps. Journal of Technology Transfer, 42(6), 1466-1486.

Ienco, D., & Meo, R. (2008). Exploration and reduction of the feature space by hierarchical clustering. Proceedings of the 2008 SIAM International conference on data mining. Society for industrial and applied mathematics.

Jonathan, C., & Meissner, D. (2018). Overcoming open innovation challenges: a contribution from foresight and foresight networks. Technology Analysis and Strategic Management, 30(6), 718-733.

Jorgenson, D. W. (2011). Innovation and productivity growth: T.W. Schultz lecture. American Journal Agricultural Economics, 92(2), 276-296.

Kellett, P. (2009). Advocacy in anthropology: Active engagement or passive scholarship? Surham anthropology Journal. 16(1), 22-31.

Kellogg, F. (2004). Logic model development guide. Battle Creek, Michigan. USA.

Lynam, J., Davis, J., Prasad, C. S., & Russo, S. L. (2009). Context is everything: A CCER on impact Assessment at the international livestock research institute (ILRI). Nairobi, Kenya.

Nicolaisen, J. (2007). Citation analysis. Annual Review of Information Science and Technology, 41(1), 609-641. https://doi.org/10.1002/aris.2007.1440410120.

North, D. (1991). Institutions. The Journal of Economic Perspectives, 5(1), 97-112.

Nyangaga, J., Romney, D., & Smutylo, T. (2006). Research beyond borders: Five cases of ILRI research outputs contributing to outcomes. ILRI Innovation Works Discussion Paper, Nairobi, Kenya. ILRI. https://hdl.handle.net/10568/56802.

Perkmann, M., & Walsh, K. (2007). University-industry relationships and open innovation: towards a research agenda. International Journal of Management Review, 9(4), 259-280.

Perkmann, M., Tartari, V., Mckelvey, M., Autio, E., Brostrom, A., D'Este, P., Fini, R., Geuna, A., Grimaldi, R., Hughes, A., Krabel, S., Kitson, M., Llerena, P., Lissoni, F., Slater, A., & Sobrero, M. (2013). Academic engagement and commercialization: A review of the literature on university-industry relations. Research Policy, 42(2), 423-442.

Shuilong, T. (2010). Development and evolution of industry-university-research cooperative innovation mode in China. Proceedings of the 7th International Conference on Innovation & Management. Wuhan University of Technology Press, Wuhan, China.

United Way. (1996). Measuring program outcomes: a practical approach. Alexandria. Va United Way of America ASIN: B000H7M5FY.

Weiss, A. P. (2007). Measuring the impact of medical research: moving from outputs to outcomes. American Journal of Psychology, 194(2), 206-214. doi.org/10.1176/appi.ajp.164.2.206.

Zaraychenko, I. A., Shinkevich, A. I., Shvetsov, M. Y., Erdyneyeva, K. G., Bordonskaya, L. A., Persidskaya, A. E., Rozhkova, S. A., & Afanasyev, A. A. (2016). Innovation networks modeling within the concept of open innovations. International Journal of Economics and Financial Issues, 6(1), 192-198.

Appendix