research paper

15 Jun research paper

Public Administration and Information Technology

Volume 10

Series Editor Christopher G. Reddick San Antonio, Texas, USA

w.jager@rug.nl

More information about this series at http://www.springer.com/series/10796

w.jager@rug.nl

Marijn Janssen • Maria A. Wimmer Ameneh Deljoo Editors

Policy Practice and Digital Science

Integrating Complex Systems, Social Simulation and Public Administration in Policy Research

2123

w.jager@rug.nl

Editors Marijn Janssen Ameneh Deljoo Faculty of Technology, Policy, and Faculty of Technology, Policy, and Management Management Delft University of Technology Delft University of Technology Delft Delft The Netherlands The Netherlands

Maria A. Wimmer Institute for Information Systems Research University of Koblenz-Landau Koblenz Germany

ISBN 978-3-319-12783-5 ISBN 978-3-319-12784-2 (eBook) Public Administration and Information Technology DOI 10.1007/978-3-319-12784-2

Library of Congress Control Number: 2014956771

Springer Cham Heidelberg New York London © Springer International Publishing Switzerland 2015 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

w.jager@rug.nl

Preface

The last economic and financial crisis has heavily threatened European and other economies around the globe. Also, the Eurozone crisis, the energy and climate change crises, challenges of demographic change with high unemployment rates, and the most recent conflicts in the Ukraine and the near East or the Ebola virus disease in Africa threaten the wealth of our societies in different ways. The inability to predict or rapidly deal with dramatic changes and negative trends in our economies and societies can seriously hamper the wealth and prosperity of the European Union and its Member States as well as the global networks. These societal and economic challenges demonstrate an urgent need for more effective and efficient processes of governance and policymaking, therewith specifically addressing crisis management and economic/welfare impact reduction.

Therefore, investing in the exploitation of innovative information and commu- nication technology (ICT) in the support of good governance and policy modeling has become a major effort of the European Union to position itself and its Member States well in the global digital economy. In this realm, the European Union has laid out clear strategic policy objectives for 2020 in the Europe 2020 strategy1: In a changing world, we want the EU to become a smart, sustainable, and inclusive economy. These three mutually reinforcing priorities should help the EU and the Member States deliver high levels of employment, productivity, and social cohesion. Concretely, the Union has set five ambitious objectives—on employment, innovation, education, social inclusion, and climate/energy—to be reached by 2020. Along with this, Europe 2020 has established four priority areas—smart growth, sustainable growth, inclusive growth, and later added: A strong and effective system of eco- nomic governance—designed to help Europe emerge from the crisis stronger and to coordinate policy actions between the EU and national levels.

To specifically support European research in strengthening capacities, in overcom- ing fragmented research in the field of policymaking, and in advancing solutions for

1 Europe 2020 http://ec.europa.eu/europe2020/index_en.htm

v

w.jager@rug.nl

vi Preface

ICT supported governance and policy modeling, the European Commission has co- funded an international support action called eGovPoliNet2. The overall objective of eGovPoliNet was to create an international, cross-disciplinary community of re- searchers working on ICT solutions for governance and policy modeling. In turn, the aim of this community was to advance and sustain research and to share the insights gleaned from experiences in Europe and globally. To achieve this, eGovPo- liNet established a dialogue, brought together experts from distinct disciplines, and collected and analyzed knowledge assets (i.e., theories, concepts, solutions, findings, and lessons on ICT solutions in the field) from different research disciplines. It built on case material accumulated by leading actors coming from distinct disciplinary backgrounds and brought together the innovative knowledge in the field. Tools, meth- ods, and cases were drawn from the academic community, the ICT sector, specialized policy consulting firms as well as from policymakers and governance experts. These results were assembled in a knowledge base and analyzed in order to produce com- parative analyses and descriptions of cases, tools, and scientific approaches to enrich a common knowledge base accessible via www.policy-community.eu.

This book, entitled “Policy Practice and Digital Science—Integrating Complex Systems, Social Simulation, and Public Administration in Policy Research,” is one of the exciting results of the activities of eGovPoliNet—fusing community building activities and activities of knowledge analysis. It documents findings of comparative analyses and brings in experiences of experts from academia and from case descrip- tions from all over the globe. Specifically, it demonstrates how the explosive growth in data, computational power, and social media creates new opportunities for policy- making and research. The book provides a first comprehensive look on how to take advantage of the development in the digital world with new approaches, concepts, instruments, and methods to deal with societal and computational complexity. This requires the knowledge traditionally found in different disciplines including public administration, policy analyses, information systems, complex systems, and com- puter science to work together in a multidisciplinary fashion and to share approaches. This book provides the foundation for strongly multidisciplinary research, in which the various developments and disciplines work together from a comprehensive and holistic policymaking perspective. A wide range of aspects for social and professional networking and multidisciplinary constituency building along the axes of technol- ogy, participative processes, governance, policy modeling, social simulation, and visualization are tackled in the 19 papers.

With this book, the project makes an effective contribution to the overall objec- tives of the Europe 2020 strategy by providing a better understanding of different approaches to ICT enabled governance and policy modeling, and by overcoming the fragmented research of the past. This book provides impressive insights into various theories, concepts, and solutions of ICT supported policy modeling and how stake- holders can be more actively engaged in public policymaking. It draws conclusions

2 eGovPoliNet is cofunded under FP 7, Call identifier FP7-ICT-2011-7, URL: www.policy- community.eu

w.jager@rug.nl

Preface vii

of how joint multidisciplinary research can bring more effective and resilient find- ings for better predicting dramatic changes and negative trends in our economies and societies.

It is my great pleasure to provide the preface to the book resulting from the eGovPoliNet project. This book presents stimulating research by researchers coming from all over Europe and beyond. Congratulations to the project partners and to the authors!—Enjoy reading!

Thanassis Chrissafis Project officer of eGovPoliNet European Commission DG CNECT, Excellence in Science, Digital Science

w.jager@rug.nl

Contents

1 Introduction to Policy-Making in the Digital Age . . . . . . . . . . . . . . . . . 1 Marijn Janssen and Maria A. Wimmer

2 Educating Public Managers and Policy Analysts in an Era of Informatics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Christopher Koliba and Asim Zia

3 The Quality of Social Simulation: An Example from Research Policy Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Petra Ahrweiler and Nigel Gilbert

4 Policy Making and Modelling in a Complex World . . . . . . . . . . . . . . . . 57 Wander Jager and Bruce Edmonds

5 From Building a Model to Adaptive Robust Decision Making Using Systems Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Erik Pruyt

6 Features and Added Value of Simulation Models Using Different Modelling Approaches Supporting Policy-Making: A Comparative Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Dragana Majstorovic, Maria A.Wimmer, Roy Lay-Yee, Peter Davis and Petra Ahrweiler

7 A Comparative Analysis of Tools and Technologies for Policy Making . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Eleni Kamateri, Eleni Panopoulou, Efthimios Tambouris, Konstantinos Tarabanis, Adegboyega Ojo, Deirdre Lee and David Price

8 Value Sensitive Design of Complex Product Systems . . . . . . . . . . . . . . . 157 Andreas Ligtvoet, Geerten van de Kaa, Theo Fens, Cees van Beers, Paulier Herder and Jeroen van den Hoven

ix

w.jager@rug.nl

x Contents

9 Stakeholder Engagement in Policy Development: Observations and Lessons from International Experience . . . . . . . . . . . . . . . . . . . . . . 177 Natalie Helbig, Sharon Dawes, Zamira Dzhusupova, Bram Klievink and Catherine Gerald Mkude

10 Values in Computational Models Revalued . . . . . . . . . . . . . . . . . . . . . . . 205 Rebecca Moody and Lasse Gerrits

11 The Psychological Drivers of Bureaucracy: Protecting the Societal Goals of an Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Tjeerd C. Andringa

12 Active and Passive Crowdsourcing in Government . . . . . . . . . . . . . . . . 261 Euripidis Loukis and Yannis Charalabidis

13 Management of Complex Systems: Toward Agent-Based Gaming for Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 Wander Jager and Gerben van der Vegt

14 The Role of Microsimulation in the Development of Public Policy . . . 305 Roy Lay-Yee and Gerry Cotterell

15 Visual Decision Support for Policy Making: Advancing Policy Analysis with Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 Tobias Ruppert, Jens Dambruch, Michel Krämer, Tina Balke, Marco Gavanelli, Stefano Bragaglia, Federico Chesani, Michela Milano and Jörn Kohlhammer

16 Analysis of Five Policy Cases in the Field of Energy Policy . . . . . . . . . 355 Dominik Bär, Maria A.Wimmer, Jozef Glova, Anastasia Papazafeiropoulou and Laurence Brooks

17 Challenges to Policy-Making in Developing Countries and the Roles of Emerging Tools, Methods and Instruments: Experiences from Saint Petersburg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 Dmitrii Trutnev, Lyudmila Vidyasova and Andrei Chugunov

18 Sustainable Urban Development, Governance and Policy: A Comparative Overview of EU Policies and Projects . . . . . . . . . . . . . 393 Diego Navarra and Simona Milio

19 eParticipation, Simulation Exercise and Leadership Training in Nigeria: Bridging the Digital Divide . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 Tanko Ahmed

w.jager@rug.nl

Contributors

Tanko Ahmed National Institute for Policy and Strategic Studies (NIPSS), Jos, Nigeria

Petra Ahrweiler EA European Academy of Technology and Innovation Assess- ment GmbH, Bad Neuenahr-Ahrweiler, Germany

Tjeerd C. Andringa University College Groningen, Institute of Artificial In- telligence and Cognitive Engineering (ALICE), University of Groningen, AB, Groningen, the Netherlands

Tina Balke University of Surrey, Surrey, UK

Dominik Bär University of Koblenz-Landau, Koblenz, Germany

Cees van Beers Faculty of Technology, Policy, and Management, Delft University of Technology, Delft, The Netherlands

Stefano Bragaglia University of Bologna, Bologna, Italy

Laurence Brooks Brunel University, Uxbridge, UK

Yannis Charalabidis University of the Aegean, Samos, Greece

Federico Chesani University of Bologna, Bologna, Italy

Andrei Chugunov ITMO University, St. Petersburg, Russia

Gerry Cotterell Centre of Methods and Policy Application in the Social Sciences (COMPASS Research Centre), University of Auckland, Auckland, New Zealand

Jens Dambruch Fraunhofer Institute for Computer Graphics Research, Darmstadt, Germany

Peter Davis Centre of Methods and Policy Application in the Social Sciences (COMPASS Research Centre), University of Auckland, Auckland, New Zealand

Sharon Dawes Center for Technology in Government, University at Albany, Albany, New York, USA

xi

w.jager@rug.nl

xii Contributors

Zamira Dzhusupova Department of PublicAdministration and Development Man- agement, United Nations Department of Economic and Social Affairs (UNDESA), NewYork, USA

Bruce Edmonds Manchester Metropolitan University, Manchester, UK

Theo Fens Faculty of Technology, Policy, and Management, Delft University of Technology, Delft, The Netherlands

Marco Gavanelli University of Ferrara, Ferrara, Italy

Lasse Gerrits Department of Public Administration, Erasmus University Rotterdam, Rotterdam, The Netherlands

Nigel Gilbert University of Surrey, Guildford, UK

Jozef Glova Technical University Kosice, Kosice, Slovakia

Natalie Helbig Center for Technology in Government, University at Albany, Albany, New York, USA

Paulier Herder Faculty of Technology, Policy, and Management, Delft University of Technology, Delft, The Netherlands

Jeroen van den Hoven Faculty of Technology, Policy, and Management, Delft University of Technology, Delft, The Netherlands

Wander Jager Groningen Center of Social Complexity Studies, University of Groningen, Groningen, The Netherlands

Marijn Janssen Faculty of Technology, Policy, and Management, Delft University of Technology, Delft, The Netherlands

Geerten van de Kaa Faculty of Technology, Policy, and Management, Delft University of Technology, Delft, The Netherlands

Eleni Kamateri Information Technologies Institute, Centre for Research & Technology—Hellas, Thessaloniki, Greece

Bram Klievink Faculty of Technology, Policy and Management, Delft University of Technology, Delft, The Netherlands

Jörn Kohlhammer GRIS, TU Darmstadt & Fraunhofer IGD, Darmstadt, Germany

Christopher Koliba University of Vermont, Burlington, VT, USA

Michel Krämer Fraunhofer Institute for Computer Graphics Research, Darmstadt, Germany

Roy Lay-Yee Centre of Methods and Policy Application in the Social Sciences (COMPASS Research Centre), University of Auckland, Auckland, New Zealand

Deirdre Lee INSIGHT Centre for Data Analytics, NUIG, Galway, Ireland

w.jager@rug.nl

Contributors xiii

Andreas Ligtvoet Faculty of Technology, Policy, and Management, Delft Univer- sity of Technology, Delft, The Netherlands

Euripidis Loukis University of the Aegean, Samos, Greece

Dragana Majstorovic University of Koblenz-Landau, Koblenz, Germany

Michela Milano University of Bologna, Bologna, Italy

Simona Milio London School of Economics, Houghton Street, London, UK

Catherine Gerald Mkude Institute for IS Research, University of Koblenz-Landau, Koblenz, Germany

Rebecca Moody Department of Public Administration, Erasmus University Rotterdam, Rotterdam, The Netherlands

Diego Navarra Studio Navarra, London, UK

Adegboyega Ojo INSIGHT Centre for Data Analytics, NUIG, Galway, Ireland

Eleni Panopoulou Information Technologies Institute, Centre for Research & Technology—Hellas, Thessaloniki, Greece

Anastasia Papazafeiropoulou Brunel University, Uxbridge, UK

David Price Thoughtgraph Ltd, Somerset, UK

Erik Pruyt Faculty of Technology, Policy, and Management, Delft University of Technology, Delft, The Netherlands; Netherlands Institute for Advanced Study, Wassenaar, The Netherlands

Tobias Ruppert Fraunhofer Institute for Computer Graphics Research, Darmstadt, Germany

Efthimios Tambouris Information Technologies Institute, Centre for Research & Technology—Hellas, Thessaloniki, Greece; University of Macedonia, Thessaloniki, Greece

Konstantinos Tarabanis Information Technologies Institute, Centre for Research & Technology—Hellas, Thessaloniki, Greece; University of Macedonia, Thessa- loniki, Greece

Dmitrii Trutnev ITMO University, St. Petersburg, Russia

Gerben van derVegt Faculty of Economics and Business, University of Groningen, Groningen, The Netherlands

Lyudmila Vidyasova ITMO University, St. Petersburg, Russia

Maria A. Wimmer University of Koblenz-Landau, Koblenz, Germany

Asim Zia University of Vermont, Burlington, VT, USA

w.jager@rug.nl

Chapter 1 Introduction to Policy-Making in the Digital Age

Marijn Janssen and Maria A. Wimmer

We are running the 21st century using 20th century systems on top of 19th century political structures. . . . John Pollock, contributing editor MIT technology review

Abstract The explosive growth in data, computational power, and social media creates new opportunities for innovating governance and policy-making. These in- formation and communications technology (ICT) developments affect all parts of the policy-making cycle and result in drastic changes in the way policies are devel- oped. To take advantage of these developments in the digital world, new approaches, concepts, instruments, and methods are needed, which are able to deal with so- cietal complexity and uncertainty. This field of research is sometimes depicted as e-government policy, e-policy, policy informatics, or data science. Advancing our knowledge demands that different scientific communities collaborate to create practice-driven knowledge. For policy-making in the digital age disciplines such as complex systems, social simulation, and public administration need to be combined.

1.1 Introduction

Policy-making and its subsequent implementation is necessary to deal with societal problems. Policy interventions can be costly, have long-term implications, affect groups of citizens or even the whole country and cannot be easily undone or are even irreversible. New information and communications technology (ICT) and models can help to improve the quality of policy-makers. In particular, the explosive growth in data, computational power, and social media creates new opportunities for in- novating the processes and solutions of ICT-based policy-making and research. To

M. Janssen (�) Faculty of Technology, Policy, and Management, Delft University of Technology, Delft, The Netherlands e-mail: m.f.w.h.a.janssen@tudelft.nl

M. A. Wimmer University of Koblenz-Landau, Koblenz, Germany

© Springer International Publishing Switzerland 2015 1 M. Janssen et al. (eds.), Policy Practice and Digital Science, Public Administration and Information Technology 10, DOI 10.1007/978-3-319-12784-2_1

w.jager@rug.nl

2 M. Janssen and M. A. Wimmer

take advantage of these developments in the digital world, new approaches, con- cepts, instruments, and methods are needed, which are able to deal with societal and computational complexity. This requires the use of knowledge which is traditionally found in different disciplines, including (but not limited to) public administration, policy analyses, information systems, complex systems, and computer science. All these knowledge areas are needed for policy-making in the digital age. The aim of this book is to provide a foundation for this new interdisciplinary field in which various traditional disciplines are blended.

Both policy-makers and those in charge of policy implementations acknowledge that ICT is becoming more and more important and is changing the policy-making process, resulting in a next generation policy-making based on ICT support. The field of policy-making is changing driven by developments such as open data, computa- tional methods for processing data, opinion mining, simulation, and visualization of rich data sets, all combined with public engagement, social media, and participatory tools. In this respect Web 2.0 and even Web 3.0 point to the specific applications of social networks and semantically enriched and linked data which are important for policy-making. In policy-making vast amount of data are used for making predictions and forecasts. This should result in improving the outcomes of policy-making.

Policy-making is confronted with an increasing complexity and uncertainty of the outcomes which results in a need for developing policy models that are able to deal with this. To improve the validity of the models policy-makers are harvesting data to generate evidence. Furthermore, they are improving their models to capture complex phenomena and dealing with uncertainty and limited and incomplete information. Despite all these efforts, there remains often uncertainty concerning the outcomes of policy interventions. Given the uncertainty, often multiple scenarios are developed to show alternative outcomes and impact. A condition for this is the visualization of policy alternatives and its impact. Visualization can ensure involvement of nonexpert and to communicate alternatives. Furthermore, games can be used to let people gain insight in what can happen, given a certain scenario. Games allow persons to interact and to experience what happens in the future based on their interventions.

Policy-makers are often faced with conflicting solutions to complex problems, thus making it necessary for them to test out their assumptions, interventions, and resolutions. For this reason policy-making organizations introduce platforms facili- tating policy-making and citizens engagements and enabling the processing of large volumes of data. There are various participative platforms developed by government agencies (e.g., De Reuver et al. 2013; Slaviero et al. 2010; Welch 2012). Platforms can be viewed as a kind of regulated environment that enable developers, users, and others to interact with each other, share data, services, and applications, enable gov- ernments to more easily monitor what is happening and facilitate the development of innovative solutions (Janssen and Estevez 2013). Platforms should provide not only support for complex policy deliberations with citizens but should also bring to- gether policy-modelers, developers, policy-makers, and other stakeholders involved in policy-making. In this way platforms provide an information-rich, interactive

w.jager@rug.nl

1 Introduction to Policy-Making in the Digital Age 3

environment that brings together relevant stakeholders and in which complex phe- nomena can be modeled, simulated, visualized, discussed, and even the playing of games can be facilitated.

1.2 Complexity and Uncertainty in Policy-Making

Policy-making is driven by the need to solve societal problems and should result in interventions to solve these societal problems. Examples of societal problems are unemployment, pollution, water quality, safety, criminality, well-being, health, and immigration. Policy-making is an ongoing process in which issues are recognized as a problem, alternative courses of actions are formulated, policies are affected, implemented, executed, and evaluated (Stewart et al. 2007). Figure 1.1 shows the typical stages of policy formulation, implementation, execution, enforcement, and evaluation. This process should not be viewed as linear as many interactions are necessary as well as interactions with all kind of stakeholders. In policy-making processes a vast amount of stakeholders are always involved, which makes policy- making complex.

Once a societal need is identified, a policy has to be formulated. Politicians, members of parliament, executive branches, courts, and interest groups may be involved in these formulations. Often contradictory proposals are made, and the impact of a proposal is difficult to determine as data is missing, models cannot

citizens

Policy formulation

Policy implementation

Policy execution

Policy enforcement and

evaluation

politicians

Policy- makers

Administrative organizations

businesses

Inspection and enforcement agencies

experts

Fig. 1.1 Overview of policy cycle and stakeholders

w.jager@rug.nl

4 M. Janssen and M. A. Wimmer

capture the complexity, and the results of policy models are difficult to interpret and even might be interpreted in an opposing way. This is further complicated as some proposals might be good but cannot be implemented or are too costly to implement. There is a large uncertainty concerning the outcomes.

Policy implementation is done by organizations other than those that formulated the policy. They often have to interpret the policy and have to make implemen- tation decisions. Sometimes IT can block quick implementation as systems have to be changed. Although policy-making is the domain of the government, private organizations can be involved to some extent, in particular in the execution of policies.

Once all things are ready and decisions are made, policies need to be executed. During the execution small changes are typically made to fine tune the policy formu- lation, implementation decisions might be more difficult to realize, policies might bring other benefits than intended, execution costs might be higher and so on. Typ- ically, execution is continually changing. Evaluation is part of the policy-making process as it is necessary to ensure that the policy-execution solved the initial so- cietal problem. Policies might become obsolete, might not work, have unintended affects (like creating bureaucracy) or might lose its support among elected officials, or other alternatives might pop up that are better.

Policy-making is a complex process in which many stakeholders play a role. In the various phases of policy-making different actors are dominant and play a role. Figure 1.1 shows only some actors that might be involved, and many of them are not included in this figure. The involvement of so many actors results in fragmentation and often actors are even not aware of the decisions made by other actors. This makes it difficult to manage a policy-making process as each actor has other goals and might be self-interested.

Public values (PVs) are a way to try to manage complexity and give some guidance. Most policies are made to adhere to certain values. Public value management (PVM) represents the paradigm of achieving PVs as being the primary objective (Stoker 2006). PVM refers to the continuous assessment of the actions performed by public officials to ensure that these actions result in the creation of PV (Moore 1995). Public servants are not only responsible for following the right procedure, but they also have to ensure that PVs are realized. For example, civil servants should ensure that garbage is collected. The procedure that one a week garbage is collected is secondary. If it is necessary to collect garbage more (or less) frequently to ensure a healthy environment then this should be done. The role of managers is not only to ensure that procedures are followed but they should be custodians of public assets and maximize a PV.

There exist a wide variety of PVs (Jørgensen and Bozeman 2007). PVs can be long-lasting or might be driven by contemporary politics. For example, equal access is a typical long-lasting value, whereas providing support for students at universities is contemporary, as politicians might give more, less, or no support to students. PVs differ over times, but also the emphasis on values is different in the policy-making cycle as shown in Fig. 1.2. In this figure some of the values presented by Jørgensen and Bozeman (2007) are mapped onto the four policy-making stages. Dependent on the problem at hand other values might play a role that is not included in this figure.

w.jager@rug.nl

1 Introduction to Policy-Making in the Digital Age 5

Policy formulation

Policy implementation

Policy execution

Policy enforcement

and evaluation

efficiency

efficiency

accountability

transparancy

responsiveness

public interest

will of the people

listening

citizen involvement

evidence-based

protection of individual rights

accountability

transparancy

evidence-based

equal access

balancing of interests

robust

honesty fair

timelessness

reliable

flexible

fair

Fig. 1.2 Public values in the policy cycle

Policy is often formulated by politicians in consultation with experts. In the PVM paradigm, public administrations aim at creating PVs for society and citizens. This suggests a shift from talking about what citizens expect in creating a PV. In this view public officials should focus on collaborating and creating a dialogue with citizens in order to determine what constitutes a PV.

1.3 Developments

There is an infusion of technology that changes policy processes at both the individual and group level. There are a number of developments that influence the traditional way of policy-making, including social media as a means to interact with the public (Bertot et al. 2012), blogs (Coleman and Moss 2008), open data (Janssen et al. 2012; Zuiderwijk and Janssen 2013), freedom of information (Burt 2011), the wisdom of the crowds (Surowiecki 2004), open collaboration and transparency in policy simulation (Wimmer et al. 2012a, b), agent-based simulation and hybrid modeling techniques (Koliba and Zia 2012) which open new ways of innovative policy-making. Whereas traditional policy-making is executed by experts, now the public is involved to fulfill requirements of good governance according to open government principles.

w.jager@rug.nl

6 M. Janssen and M. A. Wimmer

Also, the skills and capabilities of crowds can be explored and can lead to better and more transparent democratic policy decisions. All these developments can be used for enhancing citizen’s engagement and to involve citizens better in the policy-making process. We want to emphasize three important developments.

1.3.1 The Availability of Big and Open Linked Data (BOLD)

Policy-making heavily depends on data about existing policies and situations to make decisions. Both public and private organizations are opening their data for use by others. Although information could be requested for in the past, governments have changed their strategy toward actively publishing open data in formats that are readily and easily accessible (for example, European_Commission 2003; Obama 2009). Multiple perspectives are needed to make use of and stimulate new practices based on open data (Zuiderwijk et al. 2014). New applications and innovations can be based solely on open data, but often open data are enriched with data from other sources. As data can be generated and provided in huge amounts, specific needs for processing, curation, linking, visualization, and maintenance appear. The latter is often denoted with big data in which the value is generated by combining different datasets (Janssen et al. 2014). Current advances in processing power and memory allows for the processing of a huge amount of data. BOLD allows for analyzing policies and the use of these data in models to better predict the effect of new policies.

1.3.2 Rise of Hybrid Simulation Approaches

In policy implementation and execution, many actors are involved and there are a huge number of factors influencing the outcomes; this complicates the prediction of the policy outcomes. Simulation models are capable of capturing the interdepen- dencies between the many factors and can include stochastic elements to deal with the variations and uncertainties. Simulation is often used in policy-making as an instrument to gain insight in the impact of possible policies which often result in new ideas for policies. Simulation allows decision-makers to understand the essence of a policy, to identify opportunities for change, and to evaluate the effect of pro- posed changes in key performance indicators (Banks 1998; Law and Kelton 1991). Simulation heavily depends on data and as such can benefit from big and open data.

Simulation models should capture the essential aspects of reality. Simulation models do not rely heavily on mathematical abstraction and are therefore suitable for modeling complex systems (Pidd 1992). Already the development of a model can raise discussions about what to include and what factors are of influence, in this way contributing to a better understanding of the situation at hand. Furthermore, experimentation using models allows one to investigate different settings and the influence of different scenarios in time on the policy outcomes.

w.jager@rug.nl

1 Introduction to Policy-Making in the Digital Age 7

The effects of policies are hard to predict and dealing with uncertainty is a key aspect in policy modeling. Statistical representation of real-world uncertainties is an integral part of simulation models (Law and Kelton 1991). The dynamics asso- ciated with many factors affecting policy-making, the complexity associated with the interdependencies between individual parts, and the stochastic elements asso- ciated with the randomness and unpredictable behavior of transactions complicates the simulations. Computer simulations for examining, explaining, and predicting so- cial processes and relationships as well as measuring the possible impact of policies has become an important part of policy-making. Traditional models are not able to address all aspects of complex policy interactions, which indicates the need for the development of hybrid simulation models consisting of a combinatory set of models built on different modeling theories (Koliba and Zia 2012). In policy-making it can be that multiple models are developed, but it is also possible to combine various types of simulation in a single model. For this purpose agent-based modeling and simulation approaches can be used as these allow for combining different type of models in a single simulation.

1.3.3 Ubiquitous User Engagement

Efforts to design public policies are confronted with considerable complexity, in which (1) a large number of potentially relevant factors needs to be considered, (2) a vast amount of data needs to be processed, (3) a large degree of uncertainty may exist, and (4) rapidly changing circumstances need to be dealt with. Utilizing computational methods and various types of simulation and modeling methods is often key to solving these kinds of problems (Koliba and Zia 2012). The open data and social media movements are making large quantities of new data available. At the same time enhancements in computational power have expanded the repertoire of instruments and tools available for studying dynamic systems and their interdependencies. In addition, sophisticated techniques for data gathering, visualization, and analysis have expanded our ability to understand, display, and disseminate complex, temporal, and spatial information to diverse audiences. These problems can only be addressed from a complexity science perspective and with a multitude of views and contributions from different disciplines. Insights and methods of complexity science should be applied to assist policy-makers as they tackle societal problems in policy areas such as environmental protection, economics, energy, security, or public safety and health. This demands user involvement which is supported by visualization techniques and which can be actively involved by employing (serious) games. These methods can show what hypothetically will happen when certain policies are implemented.

w.jager@rug.nl

8 M. Janssen and M. A. Wimmer

1.4 Combining Disciplines in E-government Policy-Making

This new field has been shaped using various names, including e-policy-making, digital policy science, computational intelligence, digital sciences, data sciences, and policy informatics (Dawes and Janssen 2013). The essence of this field it that it is

1. Practice-driven 2. Employs modeling techniques 3. Needs the knowledge coming from various disciplines 4. It focused on governance and policy-making

This field is practice-driven by taking as a starting point the public policy problem and defining what information is relevant for addressing the problem under study. This requires understanding of public administration and policy-making processes. Next, it is a key to determine how to obtain, store, retrieve, process, model, and interpret the results. This is the field of e-participation, policy-modeling, social simulation, and complex systems. Finally, it should be agreed upon how to present and disseminate the results so that other researchers, decision-makers, and practitioners can use it. This requires in-depth knowledge of practice, of structures of public administration and constitutions, political cultures, processes and culture and policy-making.

Based on the ideas, the FP7 project EgovPoliNet project has created an inter- national community in ICT solutions for governance and policy-modeling. The “policy-making 2.0” LinkedIn community has a large number of members from dif- ferent disciplines and backgrounds representing practice and academia. This book is the product of this project in which a large number of persons from various dis- ciplines and representing a variety of communities were involved. The book shows experiences and advances in various areas of policy-making. Furthermore, it contains comparative analyses and descriptions of cases, tools, and scientific approaches from the knowledge base created in this project. Using this book, practices and knowl- edge in this field is shared among researchers. Furthermore, this book provides the foundations in this area. The covered expertise include a wide range of aspects for so- cial and professional networking and multidisciplinary constituency building along the axes of technology, participative processes, governance, policy-modeling, social simulation, and visualization. In this way eGovPoliNet has advanced the way re- search, development, and practice is performed worldwide in using ICT solutions for governance and policy-modeling.

Although in Europe the term “e-government policy” or “e-policy,” for short, is often used to refer to these types of phenomena, whereas in the USA often the term “policy informatics” is used. This is similar to that in the USA the term digital government is often used, whereas in Europe the term e-government is preferred. Policy informatics is defined as “the study of how information is leveraged and efforts are coordinated towards solving complex public policy problems” (Krishnamurthy et al. 2013, p. 367). These authors view policy informatics as an emerging research space to navigate through the challenges of complex layers of uncertainty within

w.jager@rug.nl

1 Introduction to Policy-Making in the Digital Age 9

governance processes. Policy informatics community has created Listserv called Policy Informatics Network (PIN-L).

E-government policy-making is closely connected to “data science.” Data science is the ability to find answers from larger volumes of (un)structured data (Davenport and Patil 2012). Data scientists find and interpret rich data sources, manage large amounts of data, create visualizations to aid in understanding data, build mathemat- ical models using the data, present and communicate the data insights/findings to specialists and scientists in their team, and if required to a nonexpert audience. These are activities which are at the heart of policy-making.

1.5 Overview of Chapters

In total 54 different authors were involved in the creation of this book. Some chapters have a single author, but most of the chapters have multiple authors. The authors rep- resent a wide range of disciplines as shown in Fig. 1.2. The focus has been on targeting five communities that make up the core field for ICT-enabled policy-making. These communities include e-government/e-participation, information systems, complex systems, public administration, and policy research and social simulation. The com- bination of these disciplines and communities are necessary to tackle policy problems in new ways. A sixth category was added for authors not belonging to any of these communities, such as philosophy and economics. Figure 1.3 shows that the authors are evenly distributed among the communities, although this is less with the chapter. Most of the authors can be classified as belonging to the e-government/e-participation community, which is by nature interdisciplinary.

Foundation The first part deals with the foundations of the book. In their Chap. 2 Chris Koliba and Asim Zia start with a best practice to be incorporated in public administration educational programs to embrace the new developments sketched in

EGOV

IS

Complex Systems

Public Administration and Policy Research

Social Simulation

other (philosophy, energy, economics, )

Fig. 1.3 Overview of the disciplinary background of the authors

w.jager@rug.nl

10 M. Janssen and M. A. Wimmer

this chapter. They identify two types of public servants that need to be educated. The policy informatics include the savvy public manager and the policy informatics analyst. This chapter can be used as a basis to adopt interdisciplinary approaches and include policy informatics in the public administration curriculum.

Petra Ahrweiler and Nigel Gilbert discuss the need for the quality of simulation modeling in their Chap. 3. Developing simulation is always based on certain as- sumptions and a model is as good as the developer makes it. The user community is proposed to assess the quality of a policy-modeling exercise. Communicative skills, patience, willingness to compromise on both sides, and motivation to bridge the formal world of modelers and the narrative world of policy-makers are suggested as key competences. The authors argue that user involvement is necessary in all stages of model development.

Wander Jager and Bruce Edmonds argue that due to the complexity that many social systems are unpredictable by nature in their Chap. 4. They discuss how some insights and tools from complexity science can be used in policy-making. In particular they discuss the strengths and weaknesses of agent-based modeling as a way to gain insight in the complexity and uncertainty of policy-making.

In the Chap. 5, Erik Pruyt sketches the future in which different systems modeling schools and modeling methods are integrated. He shows that elements from policy analysis, data science, machine learning, and computer science need to be combined to deal with the uncertainty in policy-making. He demonstrates the integration of various modeling and simulation approaches and related disciplines using three cases.

Modeling approaches are compared in the Chap. 6 authored by Dragana Majs- torovic, Maria A. Wimmer, Roy Lay-Yee, Peter Davis,and Petra Ahrweiler. Like in the previous chapter they argue that none of the theories on its own is able to address all aspects of complex policy interactions, and the need for hybrid simulation models is advocated.

The next chapter is complimentary to the previous chapter and includes a com- parison of ICT tools and technologies. The Chap. 7 is authored by Eleni Kamateri, Eleni Panopoulou, Efthimios Tambouris, Konstantinos Tarabanis, Adegboyega Ojo, Deirdre Lee, and David Price. This chapter can be used as a basis for tool selecting and includes visualization, argumentation, e-participation, opinion mining, simula- tion, persuasive, social network analysis, big data analytics, semantics, linked data tools, and serious games.

Social Aspects, Stakeholders and Values Although much emphasis is put on mod- eling efforts, the social aspects are key to effective policy-making. The role of values is discussed in the Chap. 8 authored by Andreas Ligtvoet, Geerten van de Kaa, Theo Fens, Cees van Beers, Paulien Herder, and Jeroen van den Hoven. Using the case of the design of smart meters in energy networks they argue that policy-makers would do well by not only addressing functional requirements but also by taking individual stakeholder and PVs into consideration.

In policy-making a wide range of stakeholders are involved in various stages of the policy-making process. Natalie Helbig, Sharon Dawes, Zamira Dzhusupova, Bram Klievink, and Catherine Gerald Mkude analyze five case studies of stakeholder

w.jager@rug.nl

1 Introduction to Policy-Making in the Digital Age 11

engagement in policy-making in their Chap. 9. Various engagement tools are dis- cussed and factors identified which support the effective use of particular tools and technologies.

The Chap. 10 investigates the role of values and trust in computational models in the policy process. This chapter is authored by Rebecca Moody and Lasse Gerrits. The authors found that a large diversity exists in values within the cases. By the authors important explanatory factors were found including (1) the role of the designer of the model, (2) the number of different actors (3) the level of trust already present, and (4) and the limited control of decision-makers over the models.

Bureaucratic organizations are often considered to be inefficient and not customer friendly. Tjeerd Andringa presents and discusses a multidisciplinary framework con- taining the drivers and causes of bureaucracy in the Chap. 11. He concludes that the reduction of the number of rules and regulations is important, but that motivating workers to understand their professional roles and to learn to oversee the impact of their activities is even more important.

Crowdsourcing has become an important policy instrument to gain access to expertise (“wisdom”) outside own boundaries. In the Chap. 12, Euripids Loukis and Yannis Charalabidis discuss Web 2.0 social media for crowdsourcing. Passive crowdsourcing exploits the content generated by users, whereas active crowdsourcing stimulates content postings and idea generation by users. Synergy can be created by combining both approaches. The results of passive crowdsourcing can be used for guiding active crowdsourcing to avoid asking users for similar types of input.

Policy, Collaboration and Games Agent-based gaming (ABG) is used as a tool to explore the possibilities to manage complex systems in the Chap. 13 by Wander Jager and Gerben van der Vegt. ABG allows for modeling a virtual and autonomous population in a computer game setting to exploit various management and leadership styles. In this way ABG contribute to the development of the required knowledge on how to manage social complex behaving systems.

Micro simulation focuses on modeling individual units and the micro-level pro- cesses that affect their development. The concepts of micro simulation are explained by Roy Lay-Yee and Gerry Cotterell in the Chap. 14. Micro simulation for pol- icy development is useful to combine multiple sources of information in a single contextualized model to answer “what if” questions on complex social phenomena.

Visualization is essential to communicate the model and the results to a variety of stakeholders. These aspects are discussed in the Chap. 15 by Tobias Ruppert, Jens Dambruch, Michel Krämer, Tina Balke, Marco Gavanelli, Stefano Bragaglia, Federico Chesani, Michela Milano, and Jörn Kohlhammer. They argue that despite the significance to use evidence in policy-making, this is seldom realized. Three case studies that have been conducted in two European research projects for policy- modeling are presented. In all the cases access for nonexperts to the computational models by information visualization technologies was realized.

w.jager@rug.nl

12 M. Janssen and M. A. Wimmer

Applications and Practices Different projects have been initiated to study the best suitable transition process towards renewable energy. In the Chap. 16 by Dominik Bär, Maria A. Wimmer, Jozef Glova, Anastasia Papazafeiropoulou,and Laurence Brooks five of these projects are analyzed and compared. They please for transferring models from one country to other countries to facilitate learning.

Lyudmila Vidyasova, Andrei Chugunov, and Dmitrii Trutnev present experiences from Russia in their Chap. 17. They argue that informational, analytical, and fore- casting activities for the processes of socioeconomic development are an important element in policy-making. The authors provide a brief overview of the history, the current state of the implementation of information processing techniques, and prac- tices for the purpose of public administration in the Russian Federation. Finally, they provide a range of recommendations to proceed.

Urban policy for sustainability is another important area which is directly linked to the first chapter in this section. In the Chap. 18, Diego Navarra and Simona Milio demonstrate a system dynamics model to show how urban policy and governance in the future can support ICT projects in order to reduce energy usage, rehabilitate the housing stock, and promote sustainability in the urban environment. This chapter contains examples of sustainable urban development policies as well as case studies.

In the Chap. 19, Tanko Ahmed discusses the digital divide which is blocking online participation in policy-making processes. Structuration, institutional and actor-network theories are used to analyze a case study of political zoning. The author recommends stronger institutionalization of ICT support and legislation for enhancing participation in policy-making and bridging the digital divide.

1.6 Conclusions

This book is the first comprehensive book in which the various development and disci- plines are covered from the policy-making perspective driven by ICT developments. A wide range of aspects for social and professional networking and multidisciplinary constituency building along the axes of technology, participative processes, gover- nance, policy-modeling, social simulation, and visualization are investigated. Policy- making is a complex process in which many stakeholders are involved. PVs can be used to guide policy-making efforts and to ensure that the many stakeholders have an understanding of the societal value that needs to be created. There is an infusion of technology resulting in changing policy processes and stakeholder involvement. Technologies like social media provides a means to interact with the public, blogs can be used to express opinions, big and open data provide input for evidence-based policy-making, the integration of various types of modeling and simulation tech- niques (hybrid models) can provide much more insight and reliable outcomes, gam- ing in which all kind of stakeholders are involved open new ways of innovative policy- making. In addition trends like the freedom of information, the wisdom of the crowds, and open collaboration changes the landscape further. The policy-making landscape is clearly changing and this demands a strong need for interdisciplinary research.

w.jager@rug.nl

1 Introduction to Policy-Making in the Digital Age 13

References

Banks J (1998) Handbook of simulation: principles, methodology, advances, applications, and practice. Wiley, New York

Bertot JC, Jaeger PT, Hansen D (2012) The impact of polices on government social media usage: Issues, challenges, and recommendations. Gov Inform Q 29:30–40

Burt E (2011) Introduction to the freedom of information special edition: emerging perspectives, critical reflections, and the need for further research. Inform Polit 16(2):91–92.

Coleman S, Moss G (2008) Governing at a distance—politicians in the blogosphere. Inform Polit 12(1–2):7–20.

Davenport TH, Patil DJ (2012) Data scientist: the sexiest job of the 21st century. Harv Bus Rev 90(10):70–76

Dawes SS, Janssen M (2013) Policy informatics: addressing complex problems with rich data, com- putational tools, and stakeholder engagement. Paper presented at the 14th annual international conference on digital government research, Quebec City, Canada

De Reuver M, Stein S, Hampe F (2013) From eparticipation to mobile participation: designing a service platform and business model for mobile participation. Inform Polit 18(1):57–73

European_Commission (2003) Directive 2003/98/EC of the European Parliament and of the coun- cil of 17 November 2003 on the re-use of public sector information. http://ec.europa.eu/ information_society/policy/psi/rules/eu/index_en.htm. Accessed 12 Dec 2012

Janssen M, Estevez E (2013) Lean government and platform-based governance—doing more with less. Gov Inform Quert 30(suppl 1):S1–S8

Janssen M, CharalabidisY, Zuiderwijk A (2012) Benefits, adoption barriers and myths of open data and open government. Inform Syst Manage 29(4):258–268

Janssen M, Estevez E, Janowski T (2014) Interoperability in big, open, and linked data— organizational maturity, capabilities, and data portfolios. Computer 47(10):26–31

Jørgensen TB, Bozeman B (2007) Public values: an inventory. Adm Soc 39(3):354–381 Koliba C, Zia A (2012) Governance Informatics: using computer simulation models to deepen

situational awareness and governance design considerations policy informatics. MIT Press, Cambridge.

Krishnamurthy R, Bhagwatwar A, Johnston EW, Desouza KC (2013) A glimpse into policy in- formatics: the case of participatory platforms that generate synthetic empathy. Commun Assoc Inform Syst 33(Article 21):365–380.

Law AM, Kelton WD (1991) Simulation modeling and analysis 2nd ed. McGraw-Hill, New York Moore MH (1995) Creating public value: strategic management in government. Harvard University

Press, Cambridge Obama B (2009) Memorandum for the Heads of executive Departments and Agencies: trans-

parency and open government. Retrieved February 21, 2013, from http://www.whitehouse.gov/ the_press_office/Transparency_and_Open_Government

Pidd M (1992) Computer simulation in management science, 3rd ed. John Wiley, Chichester Slaviero C, Maciel C, Alencar F, Santana E, Souza P (2010) Designing a platform to facilitate

the development of virtual e-participation environments. Paper presented at the ICEGOV ’10 proceedings of the 4th international conference on theory and practice of electronic governance, Beijing

Stewart JJ, Hedge DM, Lester JP (2007) Public policy: an evolutionary approach 3rd edn. Cengage Learning, Wadsworth

Stoker G (2006) Public value management: a new narrative for networked governance? Am Rev Public Adm 3(1):41–57

Surowiecki J (2004) The wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business economies, societies and nations. Doubleday

Welch EW (2012) The rise of participative technologies in government. In: Shareef MA, Archer N, Dwivedi YK, Mishra A, Pandey SK (eds) Transformational government through eGov practice: socioeconomic, cultural, and technological issues. Emerald Group Publishing Limited

w.jager@rug.nl

http://ec.europa.eu/information_society/policy/psi/rules/eu/index_en.htm.
http://ec.europa.eu/information_society/policy/psi/rules/eu/index_en.htm.
http://www.whitehouse.gov/theprotect LY1 extunderscore pressprotect LY1 extunderscore office/Transparencyprotect LY1 extunderscore andprotect LY1 extunderscore Openprotect LY1 extunderscore Government
http://www.whitehouse.gov/theprotect LY1 extunderscore pressprotect LY1 extunderscore office/Transparencyprotect LY1 extunderscore andprotect LY1 extunderscore Openprotect LY1 extunderscore Government
14 M. Janssen and M. A. Wimmer

Wimmer MA, Furdik K, Bicking M, Mach M, Sabol T, Butka P (2012a) Open collaboration in policy development: concept and architecture to integrate scenario development and formal policy modelling. In: Charalabidis Y, Koussouris S (eds) Empowering open and collaborative governance. Technologies and methods for online citizen engagement in public policy making. Springer, Berlin, pp 199–219

Wimmer MA, Scherer S, Moss S, Bicking M (2012b) Method and tools to support stakeholder engagement in policy development the OCOPOMO project. Int J Electron Gov Res (IJEGR) 8(3):98–119

Zuiderwijk A, Janssen M (2013) A coordination theory perspective to improve the use of open data in policy-making. Paper presented at the 12th conference on Electronic Government (EGOV), Koblenz

Zuiderwijk A, Helbig N, Gil-García JR, Janssen M (2014) Innovation through open data—a review of the state-of-the-art and an emerging research agenda. J Theor Appl Electron Commer Res 9(2):I–XIII.

w.jager@rug.nl

Chapter 2 Educating Public Managers and Policy Analysts in an Era of Informatics

Christopher Koliba and Asim Zia

Abstract In this chapter, two ideal types of practitioners who may use or cre- ate policy informatics projects, programs, or platforms are introduced: the policy informatics-savvy public manager and the policy informatics analyst. Drawing from our experiences in teaching an informatics-friendly graduate curriculum, we dis- cuss the range of learning competencies needed for traditional public managers and policy informatics-oriented analysts to thrive in an era of informatics. The chapter begins by describing the two different types of students who are, or can be touched by, policy informatics-friendly competencies, skills, and attitudes. Competencies ranging from those who may be users of policy informatics and sponsors of policy informatics projects and programs to those analysts designing and executing policy informatics projects and programs will be addressed. The chapter concludes with an illustration of how one Master of Public Administration (MPA) program with a policy informatics-friendly mission, a core curriculum that touches on policy infor- matics applications, and a series of program electives that allows students to develop analysis and modeling skills, designates its informatics-oriented competencies.

2.1 Introduction

The range of policy informatics opportunities highlighted in this volume will require future generations of public managers and policy analysts to adapt to the oppor- tunities and challenges posed by big data and increasing computational modeling capacities afforded by the rapid growth in information technologies. It will be up to the field’s Master of Public Administration (MPA) and Master of Public Policy (MPP) programs to provide this next generation with the tools needed to harness the wealth of data, information, and knowledge increasingly at the disposal of public

C. Koliba (�) University of Vermont, 103 Morrill Hall, 05405 Burlington, VT, USA e-mail: ckoliba@uvm.edu

A. Zia University of Vermont, 205 Morrill Hall, 05405 Burlington, VT, USA e-mail: azia@uvm.edu

© Springer International Publishing Switzerland 2015 15 M. Janssen et al. (eds.), Policy Practice and Digital Science, Public Administration and Information Technology 10, DOI 10.1007/978-3-319-12784-2_2

w.jager@rug.nl

16 C. Koliba and A. Zia

administrators and policy analysts. In this chapter, we discuss the role of policy infor- matics in the development of present and future public managers and policy analysts. Drawing from our experiences in teaching an informatics-friendly graduate curricu- lum, we discuss the range of learning competencies needed for traditional public managers and policy informatics-oriented analysts to thrive in an era of informatics. The chapter begins by describing the two different types of students who are, or can be touched by, policy informatics-friendly competencies, skills, and attitudes. Com- petencies ranging from those who may be users of policy informatics and sponsors of policy informatics projects and programs to those analysts designing and executing policy informatics projects and programs will be addressed. The chapter concludes with an illustration of how one MPA program with a policy informatics-friendly mission, a core curriculum that touches on policy informatics applications, and a series of program electives that allows students to develop analysis and modeling skills, designates its informatics-oriented competencies.

2.2 Two Types of Practitioner Orientations to Policy Informatics

Drawn from our experience, we find that there are two “ideal types” of policy infor- matics practitioner, each requiring greater and greater levels of technical mastery of analytics techniques and approaches. These ideal types are: policy informatics-savvy public managers and policy informatics analysts.

A policy informatics-savvy public manager may take on one of two possible roles relative to policy informatics projects, programs, or platforms. They may play instru- mental roles in catalyzing and implementing informatics initiatives on behalf of their organizations, agencies, or institutions. In the manner, they may work with technical experts (analysts) to envision possible uses for data, visualizations, simulations, and the like. Public managers may also be in the role of using policy informatics projects, programs, or platforms. They may be in positions to use these initiatives to ground decision making, allocate resources, and otherwise guide the performance of their organizations.

A policy informatics analyst is a person who is positioned to actually execute a policy informatics initiative. They may be referred to as analysts, researchers, modelers, or programmers and provide the technical assistance needed to analyze databases, build and run models, simulations, and otherwise construct useful and effective policy informatics projects, programs, or platforms.

To succeed in either and both roles, managers and analysts will require a certain set of skills, knowledge, or competencies. Drawing on some of the prevailing literature and our own experiences, we lay out an initial list of potential competencies for consideration.

w.jager@rug.nl

2 Educating Public Managers and Policy Analysts in an Era of Informatics 17

2.2.1 Policy Informatics-Savvy Public Managers

To successfully harness policy informatics, public managers will likely not need to know how to explicitly build models or manipulate big data. Instead, they will need to know what kinds of questions that policy informatics projects or programs can answer or not answer. They will need to know how to contract with and/or manage data managers, policy analysts, and modelers. They will need to be savvy consumers of data analysis and computational models, but not necessarily need to know how to technically execute them. Policy informatics projects, programs, and platforms are designed and executed in some ways, as any large-scale, complex project.

In writing about the stages of informatics project development using “big data,” DeSouza lays out project development along three stages: planning, execution, and postimplementation. Throughout the project life cycle, he emphasizes the role of understanding the prevailing policy and legal environment, the need to venture into coalition building, the importance of communicating the broader opportunities af- forded by the project, the need to develop performance indicators, and the importance of lining up adequate financial and human resources (2014).

Framing what traditional public managers need to know and do to effectively interface with policy informatics projects and programs requires an ability to be a “systems thinker,” an effective evaluator, a capacity to integrate informatics into performance and financial management systems, effective communication skills, and a capacity to draw on social media, information technology, and e-governance approaches to achieve common objectives. We briefly review each of these capacities below.

Systems Thinking Knowing the right kinds of questions that may be asked through policy informatics projects and programs requires public managers to possess a “sys- tems” view. Much has been written about the importance of “systems thinking” for public managers (Katz and Kahn 1978; Stacey 2001; Senge 1990; Korton 2001). Taking a systems perspective allows public managers to understand the relationship between the “whole” and the “parts.” Systems-oriented public managers will possess a level of situational awareness (Endsley 1995) that allows them to see and under- stand patterns of interaction and anticipate future events and orientations. Situational awareness allows public mangers to understand and evaluate where data are coming from, how best data are interpreted, and the kinds of assumptions being used in specific interpretations (Koliba et al. 2011). The concept of system thinking laid out here can be associated with the notion of transition management (Loorbach 2007).

Process Orientations to Public Policy The capacity to view the policy making and implementation process as a process that involves certain levels of coordination and conflict between policy actors is of critical importance for policy informatics- savvy public managers and analysts. Understanding how data are used to frame problems and policy solutions, how complex governance arrangements impact policy implementation (Koliba et al. 2010), and how data visualization can be used to

w.jager@rug.nl

18 C. Koliba and A. Zia

facilitate the setting of policy agendas and open policy windows (Kingdon 1984) is of critical importance for public management and policy analysts alike.

Research Methodologies Another basic competency needed for any public manager using policy informatics is a foundational understanding of research methods, par- ticularly quantitative reasoning and methodologies. A foundational understanding of data validity, analytical rigor and relevance, statistical significance, and the like are needed to be effective consumers of informatics. That said, traditional public man- agers should also be exposed to qualitative methods as well, refining their powers of observation, understanding how symbols, stories, and numbers are used to govern, and how data and data visualization and computer simulations play into these mental models.

Performance Management A key feature of systems thinking as applied to policy informatics is the importance of understanding how data and analysis are to be used and who the intended users of the data are (Patton 2008). The integration of policy informatics into strategic planning (Bryson 2011), performance management systems (Moynihan 2008), and ultimately woven into an organization’s capacity to learn, adapt, and evolve (Argyis and Schön 1996) are critically important in this vein. As policy informatics trends evolve, public managers will likely need to be exposed to uses of decision support tools, dashboards, and other computationally driven models and visualizations to support organizational performance.

Financial Management Since the first systemic budgeting systems were put in place, public managers have been urged to use the budgeting process as a planning and eval- uation tool (Willoughby 1918). This approach was formally codified in the 1960s with the planning–programming–budgeting (PPB) system with its focus on plan- ning, managerial, and operational control (Schick 1966) and later adopted into more contemporary approaches to budgeting (Caiden 1981). Using informative projects, programs, or platforms to make strategic resource allocation decisions is a necessary given and a capacity that effective public managers must master. Likewise, the pol- icy analyst will likely need to integrate financial resource flows and costs into their projects.

Collaborative and Cooperative Capacity Building The development and use of pol- icy informatics projects, programs, or platforms is rarely, if ever, undertaken as an individual, isolated endeavor. It is more likely that such initiatives will require interagency, interorganizational, or intergroup coordination. It is also likely that content experts will need to be partnered with analysts and programmers to com- plete tasks and execute designs. The public manager and policy analyst must both possess the capacity to facilitate collaborative management functions (O’Leary and Bingham 2009).

Basic Communication Skills This perhaps goes without saying, but the heart of any informatics project lies in the ability to effectively communicate findings and ideas through the analysis of data.

w.jager@rug.nl

2 Educating Public Managers and Policy Analysts in an Era of Informatics 19

Social Media, Information Technology, and e-Governance Awareness A final com- petency concerns public managers’ capacity to deepen their understanding of how social media, Web-based tools, and related information technologies are being em- ployed to foster various e-government, e-governance, and related initiatives (Mergel 2013). Placing policy informatics projects and programs within the context of these larger trends and uses is something that public managers must be exposed to.

Within our MPA program, we have operationalized these capacities within a four- point rubric that outlines what a student needs to do to demonstrate meeting these standards. The rubric below highlights 8 of our program’s 18 capacities. All 18 of these capacities are situated under 1 of the 5 core competencies tied to the accred- itation standards of the Network of Schools of Public Affairs and Administration (NASPAA), the professional accrediting association in the USA, and increasingly in other countries as well, for MPA and MPP programs. A complete list of these core competencies and the 18 capacities nested under them are provided in Appendix of this chapter.

The eight capacities that we have singled out as being the most salient to the role of policy informatics in public administration are provided in Table 2.1. The rubric follows a four-point scale, ranging from “does not meet standard,” “approaches standard,” “meets standard,” and “exceeds standard.”

2.2.2 Policy Informatics Analysts

A second type of practitioner to be considered is what we are referring to as a “policy informatics analyst.” When considering the kinds of competencies that policy infor- matics analysts need to be successful, we first assume that the basic competencies outlined in the prior section apply here as well. In other words, effective policy in- formatics analysts must be systems thinkers in order to place data and their analysis into context, be cognizant of current uses of decision support systems (and related platforms) to enable organizational learning, performance, and strategic planning, and possess an awareness of e-governance and e-government initiatives and how they are transforming contemporary public management and policy planning practices. In addition, policy analysts must possess a capacity to understand policy systems: How policies are made and implemented? This baseline understanding can then be used to consider the placement, purpose, and design of policy informatics projects or programs. We lay out more specific analyst capacities below.

Advanced Research Methods of Information Technology Applications In many in- stances, policy informatics analysts will need to move beyond meeting the standard. This is particularly true in the area of exceeding the public manager standards for re- search methods and utilization of information technology. It is assumed that effective policy informatics analysts will have a strong foundation in quantitative methodolo- gies and applications. To obtain these skills, policy analysts will need to move beyond basic surveys of research methods into more advanced research methods curriculum.

w.jager@rug.nl

20 C. Koliba and A. Zia

Ta bl

e 2.

1 Pu

bl ic

m an

ag er

po lic

y in

fo rm

at ic

s ca

pa ci

tie s

C ap

ac ity

D oe

s no

tm ee

ts ta

nd ar

d A

pp ro

ac he

s st

an da

rd M

ee ts

st an

da rd

E xc

ee ds

st an

da rd

C ap

ac it

y to

ap pl

y kn

ow le

dg e

of sy

st em

dy na

m ic

s an

d ne

tw or

k st

ru ct

ur es

in pu

bl ic

ad m

in is

tr at

io n

pr ac

ti ce

s

D oe

s no

tu nd

er st

an d

th e

ba si

c op

er at

io ns

of sy

st em

s an

d ne

tw or

ks ;c

an no

te xp

la in

w hy

un de

rs ta

nd in

g ca

se s

an d

co nt

ex ts

in te

rm s

of sy

st em

s an

d ne

tw or

ks is

im po

rt an

t

C an

pr ov

id e

a ba

si c

ov er

vi ew

of w

ha ts

ys te

m dy

na m

ic s

an d

ne tw

or k

st ru

ct ur

es ar

e an

d ill

us tr

at e

ho w

th ey

ar e

ev id

en t

in pa

rt ic

ul ar

ca se

s an

d co

nt ex

ts

Is ab

le to

un de

rt ak

e an

an al

ys is

of a

co m

pl ex

pu bl

ic ad

m in

is tr

at io

n is

su e,

pr ob

le m

, or

co nt

ex tu

si ng

ba si

c sy

st em

dy na

m ic

s an

d ne

tw or

k fr

am ew

or ks

C an

ap pl

y sy

st em

dy na

m ic

s an

d ne

tw or

k fr

am ew

or ks

to ex

is tin

g ca

se s

an d

co nt

ex ts

to de

ri ve

w or

ki ng

so lu

tio ns

or fe

as ib

le al

te rn

at iv

es to

pr es

si ng

ad m

in is

tr at

iv e

an d

po lic

y pr

ob le

m s

C ap

ac it

y to

ap pl

y po

li cy

st re

am s,

cy cl

es ,

sy st

em s

fo ci

up on

pa st

, pr

es en

t, an

d fu

tu re

po li

cy is

su es

,a nd

to un

de rs

ta nd

ho w

pr ob

le m

id en

ti fic

at io

n im

pa ct

s pu

bl ic

ad m

in is

tr at

io n

Po ss

es se

s lim

ite d

ca pa

ci ty

to ut

ili ze

po lic

y st

re am

s an

d po

lic y

st ag

e he

ur is

tic s

m od

el to

de sc

ri be

ob se

rv ed

ph en

om en

a. C

an is

ol at

e si

m pl

e pr

ob le

m s

fr om

so lu

tio ns

,b ut

ha s

di ffi

cu ltl

y se

pa ra

tin g

ill -s

tr uc

tu re

d pr

ob le

m s

fr om

so lu

tio ns

Po ss

es se

s so

m e

ca pa

ci ty

to ut

ili ze

po lic

y st

re am

s an

d to

de sc

ri be

po lic

y st

ag e

he ur

is tic

s m

od el

ob se

rv ed

ph en

om en

a. Po

ss es

se s

so m

e ca

pa ci

ty to

de fin

e ho

w pr

ob le

m s

ar e

fr am

ed by

di ff

er en

tp ol

ic y

ac to

rs

E m

pl oy

s a

po lic

y st

re am

s or

po lic

y st

ag e

he ur

is tic

s m

od el

ap pr

oa ch

to th

e st

ud y

of ob

se rv

ed ph

en om

en a.

C an

de m

on st

ra te

ho w

pr ob

le m

de fin

iti on

is de

fin ed

w ith

in sp

ec ifi

c po

lic y

co nt

ex ts

an d

de co

ns tr

uc tt

he re

la tio

ns hi

p be

tw ee

n pr

ob le

m de

fin iti

on s

an d

so lu

tio ns

E m

pl oy

s a

po lic

y st

re am

s or

po lic

y st

ag e

he ur

is tic

s m

od el

ap pr

oa ch

to th

e di

ag no

si s

of a

pr ob

le m

ra is

ed in

re al

-l if

e po

lic y

di le

m m

as .C

an ar

tic ul

at e

ho w

co nfl

ic ts

ov er

pr ob

le m

de fin

iti on

co nt

ri bu

te to

w ic

ke d

po lic

y pr

ob le

m s

C ap

ac it

y to

em pl

oy qu

an ti

ta ti

ve an

d qu

al it

at iv

e re

se ar

ch m

et ho

ds fo

r pr

og ra

m ev

al ua

ti on

an d

ac ti

on re

se ar

ch

Po ss

es se

s a

lim ite

d ca

pa ci

ty to

em pl

oy su

rv ey

,i nt

er vi

ew ,o

r ot

he r

so ci

al re

se ar

ch m

et ho

ds to

a fo

cu s

ar ea

.C an

ex pl

ai n

w hy

it is

im po

rt an

tt o

un de

rt ak

e pr

og ra

m or

pr oj

ec t

ev al

ua tio

n, bu

tp os

se ss

es lim

ite d

ca pa

ci ty

to ac

tu al

ly ca

rr yi

ng it

ou t

D em

on st

ra te

s a

ca pa

ci ty

to em

pl oy

su rv

ey ,i

nt er

vi ew

,o r

ot he

r so

ci al

re se

ar ch

m et

ho ds

to a

fo cu

s ar

ea an

d an

un de

rs ta

nd in

g of

ho w

su ch

da ta

an d

an al

ys is

ar e

us ef

ul in

ad m

in is

tr at

iv e

pr ac

tic e.

C an

pr ov

id e

a ra

tio na

le fo

r un

de rt

ak in

g pr

og ra

m /p

ro je

ct

C an

pr ov

id e

a pi

ec e

of or

ig in

al an

al ys

is of

an ob

se rv

ed ph

en om

en on

em pl

oy in

g on

e qu

al ita

tiv e

or qu

an tit

at iv

e m

et ho

do lo

gy ef

fe ct

iv el

y. Po

ss es

se s

ca pa

ci ty

to co

m m

is si

on a

pi ec

e of

or ig

in al

re se

ar ch

.C an

pr ov

id e

a de

ta ile

d ac

co un

tf or

ho w

a

D em

on st

ra te

s th

e ca

pa ci

ty to

un de

rt ak

e an

in de

pe nd

en t

re se

ar ch

ag en

da th

ro ug

h em

pl oy

in g

on e

or m

or e

so ci

al re

se ar

ch m

et ho

ds ar

ou nd

a to

pi c

of st

ud y

of im

po rt

an ce

to pu

bl ic

ad m

in is

tr at

io n.

C an

de m

on st

ra te

th e

su cc

es sf

ul ex

ec ut

io n

of a

pr og

ra m

or

w.jager@rug.nl

2 Educating Public Managers and Policy Analysts in an Era of Informatics 21

Ta bl

e 2.

1 (c

on tin

ue d)

C ap

ac ity

D oe

s no

tm ee

ts ta

nd ar

d A

pp ro

ac he

s st

an da

rd M

ee ts

st an

da rd

E xc

ee ds

st an

da rd

ev al

ua tio

n an

d ex

pl ai

n w

ha tt

he po

ss ib

le go

al s

an d

ou tc

om es

of su

ch an

ev al

ua tio

n m

ig ht

be

pr og

ra m

or pr

oj ec

te va

lu at

io n

pr oj

ec ts

ho ul

d be

st ru

ct ur

ed w

ith in

th e

co nt

ex to

f a

sp ec

ifi c

pr og

ra m

or pr

oj ec

t

pr oj

ec te

va lu

at io

n or

th e

su cc

es sf

ul ut

ili za

tio n

of a

pr og

ra m

or pr

oj ec

te va

lu at

io n

to im

pr ov

e ad

m in

is tr

at iv

e pr

ac tic

e

C ap

ac it

y to

ap pl

y so

un d

pe rf

or m

an ce

m ea

su re

m en

ta nd

m an

ag em

en tp

ra ct

ic es

C an

pr ov

id e

an ex

pl an

at io

n of

w hy

pe rf

or m

an ce

go al

s an

d m

ea su

re s

ar e

im po

rt an

ti n

pu bl

ic ad

m in

is tr

at io

n, bu

t ca

nn ot

ap pl

y th

is re

as on

in g

to sp

ec ifi

c co

nt ex

ts

C an

id en

tif y

th e

pe rf

or m

an ce

m an

ag em

en tc

on si

de ra

tio ns

fo r

a pa

rt ic

ul ar

si tu

at io

n or

co nt

ex t,

bu th

as lim

ite d

ca pa

ci ty

to ev

al ua

te th

e ef

fe ct

iv en

es s

of pe

rf or

m an

ce m

an ag

em en

t sy

st em

s

C an

id en

tif y

an d

an al

yz e

pe rf

or m

an ce

m an

ag em

en t

sy st

em s,

ne ed

s, an

d em

er gi

ng op

po rt

un iti

es w

ith in

a sp

ec ifi

c or

ga ni

za tio

n or

ne tw

or k

C an

pr ov

id e

ne w

in si

gh ts

in to

th e

pe rf

or m

an ce

m an

ag em

en t

ch al

le ng

es fa

ci ng

an or

ga ni

za tio

n or

ne tw

or k,

an d

su gg

es ta

lte rn

at iv

e de

si gn

an d

m ea

su re

m en

ts ce

na ri

os

C ap

ac it

y to

ap pl

y so

un d

fin an

ci al

pl an

ni ng

an d

fis ca

l re

sp on

si bi

li ty

C an

id en

tif y

w hy

bu dg

et in

g an

d so

un d

fis ca

lm an

ag em

en t

pr ac

tic es

ar e

im po

rt an

t, bu

t ca

nn ot

an al

yz e

ho w

an d/

or if

su ch

pr ac

tic es

ar e

be in

g us

ed w

ith in

sp ec

ifi c

co nt

ex ts

C an

id en

tif y

fis ca

lp la

nn in

g an

d bu

dg et

in g

pr ac

tic es

fo r

a pa

rt ic

ul ar

si tu

at io

n or

co nt

ex t,

bu th

as lim

ite d

ca pa

ci ty

to ev

al ua

te th

e ef

fe ct

iv en

es s

of a

fin an

ci al

m an

ag em

en ts

ys te

m

C an

id en

tif y

an d

an al

yz e

fin an

ci al

m an

ag em

en t

sy st

em s,

ne ed

s, an

d em

er gi

ng op

po rt

un iti

es w

ith in

a sp

ec ifi

c or

ga ni

za tio

n or

ne tw

or k

C an

pr ov

id e

ne w

in si

gh ts

in to

th e

fin an

ci al

m an

ag em

en t

ch al

le ng

es fa

ci ng

an or

ga ni

za tio

n or

ne tw

or k,

an d

su gg

es ta

lte rn

at iv

e de

si gn

an d

bu dg

et in

g sc

en ar

io s

C ap

ac it

y to

ac hi

ev e

co op

er at

io n

th ro

ug h

pa rt

ic ip

at or

y pr

ac ti

ce s

C an

ex pl

ai n

w hy

it is

im po

rt an

tf or

pu bl

ic ad

m in

is tr

at or

s to

be op

en an

d re

sp on

si ve

pr ac

tit io

ne rs

in a

va gu

e or

ab st

ra ct

w ay

,b ut

ca nn

ot pr

ov id

e sp

ec ifi

c ex

pl an

at io

ns or

ju st

ifi ca

tio ns

ap pl

ie d

to pa

rt ic

ul ar

co nt

ex ts

C an

id en

tif y

in st

an ce

s in

sp ec

ifi c

ca se

s or

co nt

ex ts

w he

re a

pu bl

ic ad

m in

is tr

at or

de m

on st

ra te

d or

fa ile

d to

de m

on st

ra te

in cl

us iv

e pr

ac tic

es

C an

de m

on st

ra te

ho w

in cl

us iv

e pr

ac tic

es an

d co

nfl ic

t m

an ag

em en

tl ea

ds to

co op

er at

io n

fo r

fo rm

in g

co al

iti on

s an

d co

lla bo

ra tiv

e pr

ac tic

es

C an

or ch

es tr

at e

an y

of th

e fo

llo w

in g:

co al

iti on

bu ild

in g

ac ro

ss un

its ,o

rg an

iz at

io ns

,o r

in st

itu tio

ns ,e

ff ec

tiv e

te am

w or

k, an

d/ or

co nfl

ic t

m an

ag em

en t

w.jager@rug.nl

22 C. Koliba and A. Zia

Ta bl

e 2.

1 (c

on tin

ue d)

C ap

ac ity

D oe

s no

tm ee

ts ta

nd ar

d A

pp ro

ac he

s st

an da

rd M

ee ts

st an

da rd

E xc

ee ds

st an

da rd

C ap

ac it

y to

un de

rt ak

e hi

gh qu

al it

y or

al ,

w ri

tt en

co m

m un

ic at

io n

D em

on st

ra te

s so

m e

ab ili

ty to

ex pr

es s

id ea

s ve

rb al

ly an

d in

w ri

tin g.

L ac

ks co

ns is

te nt

ca pa

ci ty

to pr

es en

ta nd

w ri

te

Po ss

es se

s th

e ca

pa ci

ty to

w ri

te do

cu m

en ts

th at

ar e

fr ee

of gr

am m

at ic

al er

ro rs

an d

ar e

or ga

ni ze

d in

a cl

ea r

an d

ef fic

ie nt

m an

ne r.

Po ss

es se

s th

e ca

pa ci

ty to

pr es

en ti

de as

in a

pr of

es si

on al

m an

ne r.

Su ff

er s

fr om

a la

ck of

co ns

is te

nc y

in th

e pr

es en

ta tio

n of

m at

er ia

la nd

ex pr

es si

on or

or ig

in al

id ea

s an

d co

nc ep

ts

Is ca

pa bl

e of

co ns

is te

nt ly

ex pr

es si

ng id

ea s

ve rb

al ly

an d

in w

ri tin

g in

a pr

of es

si on

al m

an ne

r th

at co

m m

un ic

at es

m es

sa ge

s to

in te

nd ed

au di

en ce

s

C an

de m

on st

ra te

so m

e in

st an

ce s

in w

hi ch

ve rb

al an

d w

ri tte

n co

m m

un ic

at io

n ha

s pe

rs ua

de d

ot he

rs to

ta ke

ac tio

n

C ap

ac it

y to

un de

rt ak

e hi

gh qu

al it

y el

ec tr

on ic

al ly

m ed

ia te

d co

m m

un ic

at io

n an

d ut

il iz

e in

fo rm

at io

n sy

st em

s an

d m

ed ia

to ad

va nc

e ob

je ct

iv es

C an

ex pl

ai n

w hy

in fo

rm at

io n

te ch

no lo

gy is

im po

rt an

tt o

co nt

em po

ra ry

w or

kp la

ce s

an d

pu bl

ic ad

m in

is tr

at io

n en

vi ro

nm en

ts .P

os se

ss es

di re

ct ex

pe ri

en ce

w ith

in fo

rm at

io n

te ch

no lo

gy ,b

ut lit

tle un

de rs

ta nd

in g

fo r

ho w

IT in

fo rm

s pr

of es

si on

al pr

ac tic

e

C an

id en

tif y

in st

an ce

s in

sp ec

ifi c

ca se

s or

co nt

ex tw

he re

a pu

bl ic

ad m

in is

tr at

or su

cc es

sf ul

ly or

un su

cc es

sf ul

ly de

m on

st ra

te d

a ca

pa ci

ty to

us e

IT to

fo st

er in

no va

tio n,

im pr

ov e

se rv

ic es

,o r

de ep

en ac

co un

ta bi

lit y.

A na

ly si

s at

th is

le ve

li s

re le

ga te

d to

de sc

ri pt

io ns

an d

th in

an al

ys is

C an

id en

tif y

ho w

IT im

pa ct

s w

or kp

la ce

s an

d pu

bl ic

po lic

y. C

an di

ag no

se pr

ob le

m s

as so

ci at

ed w

ith IT

to ol

s, pr

oc ed

ur es

,a nd

us es

D em

on st

ra te

s a

ca pa

ci ty

to vi

ew IT

in te

rm s

of sy

st em

s de

si gn

.I s

ca pa

bl e

of w

or ki

ng w

ith IT

pr of

es si

on al

s in

id en

tif yi

ng ar

ea s

of ne

ed fo

r IT

up gr

ad es

,I T

pr oc

ed ur

es ,

an d

IT us

es in

re al

se tti

ng

IT in

fo rm

at io

n te

ch no

lo gy

w.jager@rug.nl

2 Educating Public Managers and Policy Analysts in an Era of Informatics 23

Competencies in advanced quantitative methods in which students learn to clean and manage large databases, perform advanced statistical tests, develop linear regression models to describe causal relationship, and the like are needed. Capacity to work across software platforms such as Excel, Statistical Package for the Social Sciences (SPSS), Analytica, and the like are important. Increasingly, the capacity to triangu- late different methods, including qualitative approaches such as interviews, focus groups, participant observations is needed.

Data Visualization and Design Not only must analysts be aware of how these meth- ods and decision support platforms may be used by practitioners but also they must know how to design and implement them. Therefore, we suggest that policy infor- matics analysts be exposed to design principles and how they may be applied to decision support systems, big data projects, and the like. Policy informatics analysts will need to understand and appreciate how data visualization techniques are being employed to “tell a story” through data.

Figure 2.1 provides an illustration of one student’s effort to visualize campaign donations to state legislatures from the gas-extraction (fracking) industry undertaken by a masters student, Jeffery Castle for a system analysis and strategic management class taught by Koliba.

Castle’s project demonstrates the power of data visualization to convey a central message drawing from existing databases. With a solid research methods background and exposure to visualization and design principles in class, he was able to develop an insightful policy informatics project.

Basic to Advanced Programming Language Skills Arguably, policy informatics ana- lysts will possess a capacity to visualize and present data in a manner that is accessible. Increasingly, web-based tools are being used to design user interfaces. Knowledge of JAVA and HTML are likely most helpful in these regards. In some instances, original programs and models will need to be written through the use of program- ming languages such as Python, R, C++, etc. The extent to which existing software programs, be they open source or proprietary, provide enough utility to execute pol- icy informatics projects, programs, or platforms is a continuing subject of debate within the policy informatics community. Exactly how much and to what extent spe- cific programming languages and software programs are needing to be mastered is a standing question. For the purposes of writing this chapter, we rely on our current baseline observations and encourage more discussion and debate about the range of competencies needed by successful policy analysts.

Basic to More Advanced Modeling Skills More advanced policy informatics analysts will employ computational modeling approaches that allow for the incorporation of more complex interactions between variables. These models may be used to capture systems as dynamic, emergent, and path dependent. The outputs of these models may allow for scenario testing through simulation (Koliba et al. 2011). With the advancement of modeling software, it is becoming easier for analysts to develop system dynamics models, agent-based models, and dynamic networks designed to simulate the features of complex adaptive systems. In addition, the ability to manage and store data and link or wrap databases is often necessary.

w.jager@rug.nl

24 C. Koliba and A. Zia

Fig. 2.1 Campaign contributions to the Pennsylvania State Senate and party membership. The goal of this analysis is to develop a visualization tool to translate publically available campaign contribution information into an easily accessible, visually appealing, and interactive format. While campaign contribution data are filed and available to the public through the Pennsylvania Department of State, it is not easily synthesized. This analysis uses a publically available database that has been published on marcellusmoney.org. In order to visualize the data, a tool was used that allows for the creation of a Sankey diagram that is able to be manipulated and interacted within an Internet browser. A Sankey diagram visualizes the magnitude of flow between the nodes of a network (Castle 2014)

The ability of analysts to draw on a diverse array of methods and theoretical frameworks to envision and create models is of critical importance. Any potential policy informatics project, program, or platform will be enabled or constrained by the modeling logic in place. With a plurality of tools at one’s disposal, policy informatics analysts will be better positioned to design relevant and legitimate models.

w.jager@rug.nl

2 Educating Public Managers and Policy Analysts in an Era of Informatics 25

Fig. 2.2 End-stage renal disease (ESRD) system dynamics population model. To provide clinicians and health care administrators with a greater understanding of the combined costs associated with the many critical care pathways associated with ESRD, a system dynamics model was designed to simulate the total expenses of ESRD treatment for the USA, as well as incidence and mortality rates associated with different critical care pathways: kidney transplant, hemodialysis, peritoneal dialysis, and conservative care. Calibrated to US Renal Data System (USRDS) 2013 Annual and Historical Data Report and the US Census Bureau for the years 2005–2010, encompassing all ESRD patients under treatment in the USA from 2005 to 2010, the ESRD population model predicts the growth and costs of ESRD treatment type populations using historical patterns. The model has been calibrated against the output of the USRDS’s own prediction for the year 2020 and also tested by running his- toric scenarios and comparing the output to existing data. Using a web interface designed to allow users to alter certain combinations of parameters, several scenarios are run to project future spending, incidence, and mortalities if certain combinations of critical care pathways are pursued. These sce- narios include: a doubling of kidney donations and transplant rates, a marked increase in the offering of peritoneal dialysis, and an increase in conservative care routes for patients over 65. The results of these scenario runs are shared, demonstrating sizable cost savings and increased survival rates. Implications of clinical practice, public policy, and further research are drawn (Fernandez 2013)

Figure 2.2 provides an illustration of Luca Fernandez’s system dynamics model of critical care pathways for end-stage renal disease (ESRD). Fernandez took Koliba’s system analysis and strategic management course and Zia’s decision-making model- ing course. This model, constructed using the proprietary software, AnyLogic, was initially constructed as a project in Zia’s course.

Castle and Fernandez’s projects illustrate how master’s-level students with an eye toward becoming policy informatics analysts can build skills and capacities to develop useful informatics projects that can guide policy and public management. They were guided to this point by taking advanced courses designed explicitly with policy informatics outcomes in mind.

w.jager@rug.nl

26 C. Koliba and A. Zia

Policy Informatics Analyst Informatics-Savvy Public

•Advanced research methods •Data visualization and design techniques •Basic to advanced modeling software skills •Basic to advanced programming language(s) •Systems thinking •Basic understanding of research methods •Knowledge of how to integrate informatics within performance management •Knowledge of how to integrate inofrmatics within financial systems•Effecive written communication •Effective usese of social media / e-governance approaches

Fig. 2.3 The nested capacities of informatics-savvy public managers and policy informatics analysts

Figure 2.3 illustrates how the competencies of the two different ideal types of policy informatics practitioners are nested inside of one another. A more complete list of competencies that are needed for the more advanced forms of policy analy- sis will need to emerge through robust exchanges between the computer sciences, organizational sciences, and policy sciences. These views will likely hinge on as- sumptions about the sophistication of the models to be developed. A key question here concerning the types of models to be built is: Can adequate models be built using existing software or is original programming needed or desired? Ideally, ad- vanced policy analysts undertaking policy informatics projects are “programmers with a public service motivation.”

2.3 Applications to Professional Masters Programs

Professional graduate degree programs have steadily moved toward emphasizing the importance of the mission of particular graduate programs in determining the optimal curriculum to suit the learning needs of it students. As a result, clear definitions of the learning outcomes and the learning needs of particular student communities are defined. Some programs may seek to serve regional or local needs of the government and nonprofit sector, while others may have a broader reach, preparing students to work within federal or international level governments and nonprofits.

In addition to geographic scope, accredited MPA and MPP programs may have specific areas of concentration. Some programs may focus on preparing public man- agers who are charged with managing resources, making operational, tactical, and

w.jager@rug.nl

2 Educating Public Managers and Policy Analysts in an Era of Informatics 27

strategic decisions and, overall, administering to the day-to-day needs of a govern- ment or nonprofit organization. Programs may also focus on training policy analysts who are responsible for analyzing policies, policy alternatives, problem definition, and the like. Historically, the differences between public management and policy analysis have distinguished the MPA degree from the MPP degree. However, recent studies of NASPAA-accredited programs have found that the lines between MPA and MPP programs are increasingly blurred (Hur and Hackbart 2009). The relationship between public management and policy analysis matters to those interested in policy informatics because these distinctions drive what policy informatics competencies and capacities are covered within a core curriculum, and what competencies and capacities are covered within a suite of electives or concentrations.

Competency-based assessments are increasingly being used to evaluate and de- sign curriculum. Drawing on the core tenants of adult learning theory and practice, competency-based assessment involves the derivation of specific skills, knowledge, or attitudes that an adult learner must obtain in order to successfully complete a course of study or degree requirement. Effective competency-based graduate pro- grams call on students to demonstrate a mastery of competencies through a variety of means. Portfolio development, test taking, and project completion are common applications. Best practices in competency-based education assert that curriculum be aligned with specific competencies as much as possible.

By way of example, the University ofVermont’s MPA Program has had a “systems thinking” focus since it was first conceived in the middle 1980s. Within the last 10 years, the two chapter coauthors, along with several core faculty who have been associated with the program since its inception, have undertaken an effort to refine its mission based on its original systems-focused orientation.

As of 2010, the program mission was refined to read:

Our MPA program is a professional interdisciplinary degree that prepares pre and in-service leaders, managers and policy analysts by combining the theoretical and practical founda- tions of public administration focusing on the complexity of governance systems and the democratic, collaborative traditions that are a hallmark of Vermont communities.

The mission was revised to include leaders and managers, as well as policy analysts. A theory-practice link was made explicit. The phrase, “complexity of governance systems” was selected to align with a commonly shared view of contemporary gover- nance as a multisectoral and multijurisdictional context. Concepts such as bounded rationality, social complexity, the importance of systems feedback, and path de- pendency are stressed throughout the curriculum. The sense of place found within the State of Vermont was also recognized and used to highlight the high levels of engagement found within the program.

The capacities laid out in Table 2.1 have been mapped to the program’s core curriculum. The program’s current core is a set of five courses: PA 301: Foundations of Public Administration; PA 302: Organizational Behavior and Change; PA 303: Research Methods; PA 305: Public and Nonprofit Budgeting and Finance and PA 306: Policy Systems. In addition, all students are required to undertake a three- credit internship and a three-credit Capstone experience in which they construct a

w.jager@rug.nl

28 C. Koliba and A. Zia

final learning portfolio. It is within this final portfolio that students are expected to provide evidence of meeting or exceeding the standard. An expanded rubric of all 18 capacities is used by the students to undertake their own self-assessment. These assessments are judged against the Capstone instructor’s evaluation.

In 2009, the MPA faculty revised the core curriculum to align with the core competencies. Several course titles and content were revised to align with these competencies and the overall systems’ focus of our mission. The two core courses taught by the two coauthors, PA 301 and PA 306, are highlighted here.

2.4 PA 301: Foundations of Public Administration

Designed as a survey of the prevailing public administration literature during the past 200 plus years, Foundations of Public Administration is arranged across a continuum of interconnected themes and topics that are to be addressed in more in-depth in other courses and is described in the syllabus in the following way:

This class is designed to provide you with an overview of the field of public administra- tion. You will explore the historical foundations, the major theoretical, organizational, and political breakthroughs, and the dynamic tensions inherent to public and nonprofit sector administration. Special attention will be given to problems arising from political imperatives generated within a democratic society.

Each week a series of classic and contemporary texts are read and reviewed by the students. In part, to fill a noticeable void in the literature, the authors co-wrote, along with Jack Meek, a book on governance networks called: Governance Networks in Public Administration and Public Policy (Koliba et al. 2010). This book is required reading. Students are also asked to purchase Shafritz and Hyde’s edited volume, Classics of Public Administration.

Current events assignments offered through blog posts are undertaken. Weekly themes include: the science and art of administration; citizens and the administra- tive state; nonprofit, private, and public sector differences; governance networks; accountability; and performance management.

During the 2009 reforms of the core curriculum, discrete units on governance networks and performance management were added to this course. Throughout the entire course, a complex systems lens is employed to describe and analyze gover- nance networks and the particular role that performance management systems play in providing feedback to governance actors. Students are exposed to social network and system dynamics theory, and asked to apply these lenses to several written cases taken from the Electronic Hallway. A unit on performance management systems and their role within fostering organizational learning are provided along with readings and examples of decision support tools and dashboard platforms currently in use by government agencies.

Across many units, including units on trends and reforms, ethical and reflective leadership, citizens and the administrative state, and accountability, the increasing use of social media and other forms of information technology are discussed. Trends

w.jager@rug.nl

2 Educating Public Managers and Policy Analysts in an Era of Informatics 29

shaping the “e-governance” and “e-government” movements serve as a major focus on current trends. In addition, students are exposed to current examples of data visualizations and open data platforms and asked to consider their uses.

2.5 PA 306: Policy Systems

Policy Systems is an entry-level graduate policy course designed to give the MPA student an overview of the policy process. In 2009, the course was revised to reflect a more integrated systems focus. The following text provides an overview of the course:

In particular, the emphasis is placed upon meso-, and macro-scale policy system frame- works and theories, such as InstitutionalAnalysis and Development Framework, the Multiple Streams Framework; Social Construction and Policy Design; the Network Approach; Punc- tuated Equilibrium Theory; the Advocacy Coalition Framework; Innovation and Diffusion Models and Large-N Comparative Models. Further, students will apply these micro-, meso- and macro-scale theories to a substantive policy problem that is of interest to a community partner, which could be a government agency or a non-profit organization. These policy problems may span, or even cut across, a broad range of policy domains such as (included but not limited to) economic policy, food policy, environmental policy, defense and foreign policy, space policy, homeland security, disaster and emergency management, social policy, transportation policy, land-use policy and health policy.

The core texts for this class are Elinor Ostrom’s, Understanding Institutional Di- versity, Paul Sabatier’s edited volume, Theories of the Policy Process, and Deborah Stone’s Policy Paradox: The Art of Political Decision-Making. The course itself is staged following a micro, to meso, to macro level scale of policy systems framework. A service-learning element is incorporated. Students are taught to view the policy process through a systems lens. Zia employs examples of policy systems models us- ing system dynamics (SD), agent-based modeling (ABM), social network analysis (SNA), and hybrid approaches throughout the class. By drawing on Ostrom, Sabatier, and other meso level policy processes as a basis, students are exposed to a number of “complexity-friendly” theoretical policy frameworks (Koliba and Zia 2013). Appre- ciating the value of these policy frameworks, students are provided with heuristics for understanding the flow of information across a system. In addition, students are shown examples of simulation models of different policy processes, streams, and systems.

In addition to PA 301 and PA 306, students are also provided an in-depth ex- ploration of organization theory in PA 302 Organizational Behavior and Change that is taught through an organizational psychology lens that emphasizes the role of organizational culture and learning. “Soft systems” approaches are applied. PA 303 Research Methods for Policy Analysis and Program Evaluation exposes students to a variety of research and program evaluation methodologies with a particular focus on quantitative analysis techniques. Within PA 305 Public and Nonprofit Budgeting and Finance, students are taught about evidence-based decision-making and data management.

w.jager@rug.nl

30 C. Koliba and A. Zia

By completing the core curriculum, students are exposed to some of the founda- tional competencies needed to use and shape policy informatics projects. However, it is not until students enroll in one of the several electives, that more explicit policy informatics concepts and applications are taught. Two of these elective courses are highlighted here. A third, PA 311 Policy Analysis, also exposes students to policy analyst capacities, but is not highlighted here.

2.6 PA 308: Decision-Making Models

A course designated during the original founding of the University of Vermont (UVM)-MPA Program, PA 308: Decision-Making Models offers students with a more advanced look at decision-making theory and modeling. The course is described by Zia in the following manner:

In this advanced graduate level seminar, we will explore and analyze a wide range of norma- tive, descriptive and prescriptive decision making models. This course focuses on systems level thinking to impart problem-solving skills in complex decision-making contexts. Deci- sion making problems in the real-world public policy, business and management arenas will be analyzed and modeled with different tools developed in the fields of Decision Analysis, Behavioral Sciences, Policy Sciences and Complex Systems. The emphasis will be placed on imparting cutting edge skills to enable students to design and implement multiple criteria decision analysis models, decision making models under risk and uncertainty and computer simulation models such as Monte Carlo simulation, system dynamic models, agent based models, Bayesian decision making models, participatory and deliberative decision making models, and interactive scenario planning approaches. AnyLogic version 6.6 will be made available to the students for working with some of these computer simulation models.

2.7 PA 317: Systems Analysis and Strategic Management

Another course designate during the early inception of the program, systems analysis and strategic management is described by Koliba in the course syllabus as follows:

This course combines systems and network analysis with organizational learning theory and practices to provide students with a heightened capacity to analyze and effectively operate in complex organizations and networks. The architecture for the course is grounded in many of the fundamental conceptual frameworks found in network, systems and complexity analysis, as well as some of the fundamental frameworks employed within the public administration and policy studies fields. In this course, strategic management and systems analysis are linked together through the concept of situational awareness and design principles. Several units focusing on teaching network analysis tools using UCINet have been incorporated.

One of the key challenges to offering these informatics-oriented electives lies in the capacities that the traditional MPA students possess to thrive within them. Increas- ingly, these elective courses are being populated by doctoral and master of science students looking to apply what they are learning to their dissertations or thesis. Our MPA program offers a thesis option and we have had some success with these more

w.jager@rug.nl

2 Educating Public Managers and Policy Analysts in an Era of Informatics 31

professionally oriented students undertaking high quality informatics focused thesis. Our experience begs a larger question pertaining to the degree to which the baseline informatics-savvy public manager capacities lead into more complex policy analysts competencies associated with the actual design and construction of policy informatics projects, programs, and platforms.

Table 2.2 provides an overview of where within the curriculum certain policy informatics capacities are covered. When associated with the class, students are exposed to the uses of informatics projects, programs, or platforms or provided opportunities for concrete skill development.

The University of Vermont context is one that can be replicated in other programs. The capacity of the MPA program to offer these courses hinges on the expertise of two faculties who teach in the core and these two electives. With additional re- sources, a more advanced curriculum may be pursued, one that pursues closer ties with the computer science department (Zia has a secondary appointment) around curricular alignment. Examples of more advanced curriculum to support the devel- opment of policy informatics analysts may be found at such institutions as Carnegie Mellon University, Arizona State University, George Mason University, University at Albany, Delft University of Technology, Massachusetts Institute of Technology, among many others. The University of Vermont case suggests, however, that pol- icy informatics education can be integrated into the main stream with relatively low resource investments leveraged by strategic relationships with other disciplines and core faculty with the right skills, training, and vision.

2.8 Conclusion

It is difficult to argue that with the advancement of high speed computing, the dig- itization of data and the increasing collaboration occurring around the development of informatics projects, programs, and platforms, that the educational establishment, particularly at the professional master degree levels, will need to evolve. This chap- ter lays out a preliminary look at some of the core competencies and capacities that public managers and policy analysts will need to lead the next generation of policy informatics integration.

w.jager@rug.nl

32 C. Koliba and A. Zia

Table 2.2 Policy informatics capacities covered within the UVM-MPA program curriculum

Course title Policy informatics-savvy public management capacities covered

Policy informatics analysis capacities covered

PA 301: Foundations of public administration

Systems thinking Policy as process Performance management Financial management Basic communication Social media/IT/e-governance Collaborative–cooperative capacity building

Data visualization and design

PA 306: Policy systems Systems thinking Policy as process Basic communication

Basic modeling skills

PA 302: Organizational behavior and change

Systems thinking Basic communication Collaborative–cooperative capacity building

PA 303: Research methods for policy analysis and program evaluation

Research methods Basic communication

Data visualization and design

PA 305: Public and nonprofit budgeting and finance

Financial management Performance management Basic communication

PA 308: Decision-making modeling

Systems thinking Policy as process Research methods Performance management Social media/IT/e-governance

Advanced research methods Data visualization and design techniques Basic modeling skills

PA 311: Policy analysis Systems thinking Policy as process Research methods Performance management Basic communication

Advanced research methods Data visualization and design Basic modeling skills

PA 317: Systems analysis and strategic analysis

Systems thinking Policy as process Research methods Performance management Collaborative–cooperative capacity building Basic communication Social media/IT/e-governance

Data visualization and design Basic modeling skills

w.jager@rug.nl

2 Educating Public Managers and Policy Analysts in an Era of Informatics 33

2.9 Appendix A: University of Vermont’s MPA Program Learning Competencies and Capacities

NASPAA core standard UVM-MPA learning capacity

To lead and manage in public governance

Capacity to understand accountability and democratic theory

Capacity to manage the lines of authority for public, private, and nonprofit collaboration, and to address sectorial differences to overcome obstacles

Capacity to apply knowledge of system dynamics and network structures in PA practice

Capacity to carry out effective policy implementation

To participate in and contribute to the policy process

Capacity to apply policy streams, cycles, systems foci upon past, present, and future policy issues, and to understand how problem identification impacts public administration

Capacity to conduct policy analysis/evaluation

Capacity to employ quantitative and qualitative research methods for program evaluation and action research

To analyze, synthesize, think critically, solve problems, and make decisions

Capacity to initiate strategic planning, and apply organizational learning and development principles

Capacity to apply sound performance measurement and management practices

Capacity to apply sound financial planning and fiscal responsibility

Capacity to employ quantitative and qualitative research methods for program evaluation and action research

To articulate and apply a public service perspective

Capacity to understand the value of authentic citizen participation in PA practice

Capacity to understand the value of social and economic equity in PA practices

Capacity to lead in an ethical and reflective manner

Capacity to achieve cooperation through participatory practices

To communicate and interact productively with a diverse and changing workforce and citizenry

Capacity to undertake high quality oral, written, and electronically mediated communication and utilize information systems and media to advance objectives

Capacity to appreciate the value of pluralism, multiculturalism, and cultural diversity

Capacity to carry out effective human resource management

Capacity to undertake high quality oral, written, and electronically mediated communication and utilize information systems and media to advance objectives

NASPAA Network of Schools of Public Affairs and Administration, UVM University of Vermont, MPA Master of Public Administration, PA Public administration

w.jager@rug.nl

34 C. Koliba and A. Zia

References

Argyis C, Schön DA (1996) Organizational learning II: theory, method, and practice. Addison- Wesley, Reading

Bryson J (2011) Strategic planning for public and nonprofit organizations: a guide to strengthening and sustaining organizational achievement. Jossey-Bass, San Francisco

Caiden N (1981) Public budgeting and finance. Blackwell, New York Castle J (2014) Visualizing natural gas industry contributions in Pennsylvania Government, PA 317

final class project Desouza KC (2014) Realizing the promise of big data: implementing big data projects. IBM Center

for the Business of Government, Washington, DC Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Hum Fact 37(1):32–

64 Fernandez L (2013) An ESRD system dynamics population model for the United States. Final

project for PA 308 Hur Y, Hackbart M (2009) MPA vs. MPP: a distinction without a difference? J Public Aff Educ

15(4):397–424 Katz D, Khan R (1978) The social psychology of organizations. Wiley, New York Kingdon J (1984) Agendas, alternatives, and public policies. Harper Collins, New York Koliba C, Zia A (2013) Complex systems modeling in public administration and policy studies:

challenges and opportunities for a meta-theoretical research program. In: Gerrits L, Marks PK (eds) COMPACT I: public administration in complexity. Emergent, Litchfield Park

Koliba C, Meek J, Zia A (2010) Governance networks in public administration and public policy. CRC, Boca Raton

Koliba C, Zia A, Lee B (2011) Governance informatics: utilizing computer simulation models to manage complex governance networks. Innov J Innov Publ Sect 16(1):1–26 (Article 3). (http://www.innovation.cc/scholarly-style/koliba_governance_informaticsv16i1a3.pdf)

Korton DC (2001) The management of social transformation. In: Stivers C (ed) Democracy, bureaucracy, and the study of administration. Westview, Boulder, pp 476–497

Loorbach D (2007) Transition management: new modes of governance for sustainable development. International Books, Ultrecht

Mergel I (2013) Social media adoption and resulting tactics in the U.S. federal government. Gov Inf Quart 30(2):123–130

Moynihan DP (2008) The dynamics of performance management: constructing information and reform. Georgetown University Press, Washington, DC

O’Leary R, Bingham L (eds) (2009) The collaborative public manager: new ideas for the twenty-first century. Georgetown University Press, Washington, DC

Patton M (2008) Utilization-focused evaluation. Sage, New York Schick A (1966) The road to PPB: the stages of budget reform. Public Admin Rev 26(4):243–259 Senge PM (1990) The fifth discipline: the art and practice of the learning organization. Doubleday

Currency, New York Stacey RD (2001) Complex responsive processes in organizations: learning and knowledge creation.

Routledge, London Willoughby WF (1918) The movement of budgetary reform in the states. D. Appleton, New York

w.jager@rug.nl

Chapter 3 The Quality of Social Simulation: An Example from Research Policy Modelling

Petra Ahrweiler and Nigel Gilbert

Abstract This chapter deals with the assessment of the quality of a simulation. The first section points out the problems of the standard view and the constructivist view in evaluating social simulations. A simulation is good when we get from it what we originally would have liked to get from the target; in this, the evaluation of the simulation is guided by the expectations, anticipations, and experience of the community that uses it. This makes the user community view the most promising mechanism to assess the quality of a policy-modelling exercise. The second section looks at a concrete policy-modelling example to test this idea. It shows that the very first negotiation and discussion with the user community to identify their questions is highly user-driven, interactive, and iterative. It requires communicative skills, patience, willingness to compromise on both sides, and motivation to make the formal world of modellers and the narrative world of practical policy making meet. Often, the user community is involved in providing data for calibrating the model. It is not an easy issue to confirm the existence, quality, and availability of data and check for formats and database requirements. As the quality of the simulation in the eyes of the user will very much depend on the quality of the informing data and the quality of the model calibration, much time and effort need to be spent in coordinating this issue with the user community. Last but not least, the user community has to check the validity of simulation results and has to believe in their quality. Users have to be enabled to understand the model, to agree with its processes and ways to produce results, to judge similarity between empirical and simulated data, etc. Although the user community view might be the most promising, it is the most work-intensive mechanism to assess the quality of a simulation. Summarising, to trust the quality of a simulation means to trust the process that produced its results. This process includes not only the design and construction of the simulation model itself but also the whole interaction between stakeholders, study team, model, and findings.

P. Ahrweiler (�) EA European Academy of Technology and Innovation Assessment GmbH, Bad Neuenahr-Ahrweiler, Germany e-mail: Petra.Ahrweiler@ea-aw.de

N. Gilbert University of Surrey, Guildford, UK

© Springer International Publishing Switzerland 2015 35 M. Janssen et al. (eds.), Policy Practice and Digital Science, Public Administration and Information Technology 10, DOI 10.1007/978-3-319-12784-2_3

w.jager@rug.nl

36 P. Ahrweiler and N. Gilbert

Table 3.1 Comparing simulations

Caffè Nero simulation Science simulation

Target Venetian Café “Real system”

Goal Getting “the feeling” (customers) and profit (owners) from it

Getting understanding and/or predictions from it

Model By reducing the many features of a Venetian Café to a few parameters

By reducing the many features of the target to a few parameters

Question Is it a good simulation, i.e. do we get from it what we want?

Is it a good simulation, i.e. do we get from it what we want?

This chapter deals with the assessment of the quality of a simulation. After dis- cussing this issue on a general level, we apply and test the assessment mechanisms using an example from policy modelling.

3.1 Quality in Social Simulation

The construction of a scientific social simulation implies the following process: “We wish to acquire something from a target entity T. We cannot get what we want from T directly. So, we proceed indirectly. Instead of T we construct another entity M, the ‘model’, which is sufficiently similar to T that we are confident that M will deliver (or reveal) the acquired something which we want to get from T. [. . .] At a moment in time, the model has structure. With the passage of time the structure changes and that is behaviour. [. . .] Clearly we wish to know the behaviour of the model. How? We may set the model running (possibly in special sets of circumstances of our choice) and watch what it does. It is this that we refer to as‘simulation’ of the target” (quoted with slight modifications from Doran and Gilbert 1994).

We also habitually refer to “simulations” in everyday life, mostly in the sense that a simulation is “an illusory appearance that manages a reality effect” (cf. Norris 1992), or as Baudrillard put it, “to simulate is to feign to have what one hasn’t” while “substituting signs for the real” (Baudrillard 1988). In a previous publication (Ahrweiler and Gilbert 2005), we used the example of the Caffè Nero in Guildford, 50 km southwest of London, as a simulation of a Venetian café—which will serve as the “real” to illustrate this view. The purpose of the café is to “serve the best coffee north of Milan”. It tries to give the impression that you are in a real Italian café—although, most of the time, the weather outside can make the illusion difficult to maintain.

The construction of everyday simulations like Caffè Nero has some resemblance to the construction of scientific social simulations (see Table 3.1):

In both cases, we build models from a target by reducing the characteristics of the latter sufficiently for the purpose at hand; in each case, we want something from the model we cannot achieve easily from the target. In the case of Caffè Nero, we cannot simply go to Venice, drink our coffee, be happy, and return. It is too expensive and

w.jager@rug.nl

3 The Quality of Social Simulation: An Example from Research Policy Modelling 37

time-consuming. We have to use the simulation. In the case of a science simulation, we cannot get data from the real system to learn about its behaviour. We have to use the simulation.

The question, whether one or the other is a good simulation, can therefore be reformulated as: Do we get from the simulation what we constructed it for?

Heeding these similarities, we shall now try to apply evaluation methods typically used for everyday simulations to scientific simulation and vice versa. Before doing so, we shall briefly discuss the “ordinary” method of evaluating simulations called the “standard view” and its adversary, a constructivist approach asserting, “anything goes”.

3.1.1 The Standard View

The standard view refers to the well-known questions and methods of verification, namely whether the code does what it is supposed to do and whether there are any bugs, and validation, namely whether the outputs (for given inputs/parameters) resemble observations of the target, although (because the processes being modelled are stochastic and because of unmeasured factors) identical outputs are not to be expected, as discussed in detail in Gilbert and Troitzsch (1997). This standard view relies on a realist perspective because it refers to the observability of reality in order to compare the “real” with artificial data produced by the simulation.

Applying the standard view to the Caffè Nero example, we can find quantitative and sometimes qualitative measures for evaluating the simulation. Using quantitative measures of similarity between it and a “real”Venetian café, we can ask, for example,

• Whether the coffee tastes the same (by measuring, for example, a quality score at blind tasting),

• Whether the Caffè is a cool place (e.g. measuring the relative temperatures inside and outside),

• Whether the noise level is the same (using a dB meter for measuring pur- poses),whether the lighting level is the same (using a light meter), and

• Whether there are the same number of tables and chairs per square metre for the customers (counting them), and so on.

In applying qualitative measures of similarity, we can again ask:

• Whether the coffee tastes the same (while documenting what comes to mind when customers drink the coffee),

• Whether the Caffè is a “cool” place (this time meaning whether it is a fashionable place to hang out),

• Whether it is a vivid, buzzing place, full of life (observing the liveliness of groups of customers),

• Whether there is the same pattern of social relationships (difficult to opera- tionalise: perhaps by observing whether the waiters spend their time talking to the customers or to the other staff), and

w.jager@rug.nl

38 P. Ahrweiler and N. Gilbert

• Whether there is a ritual for serving coffee and whether it is felt to be the same as in a Venetian café.

The assumption lying behind these measures is that there is a “real” café and a “simulation” café and that in both of these, we can make observations. Similarly, we generally assume that the theories and models that lie at the base of science simulations are well grounded and can be validated by observation of empirical facts. However, the philosophy of science forces us to be more modest.

3.1.1.1 The Problem of Under-determination

Some philosophers of science argue that theories are under-determined by observa- tional data or experience, that is, the same empirical data may be in accord with many alternative theories. An adherent of the standard view would respond in that one important role of simulations (and of any form of model building) is to derive from theories as many testable implications as possible so that eventually validity can be assessed in a cumulative process1. Simulation is indeed a powerful tool for testing theories in that way if we are followers of the standard view.

However, the problem that theories are under-determined by empirical data can- not be solved by cumulative data gathering: it is more general and therefore more serious. The under-determination problem is not about a missing quantity of data but about the relation between data and theory. As Quine (1977) presents it: If it is possible to construct two or more incompatible theories by relying on the same set of experimental data, the choice between these theories cannot depend on “empirical facts”. Quine showed that there is no procedure to establish a relation of uniqueness between theory and data in a logically exclusive way. This leaves us with an annoying freedom: “sometimes, the same datum is interpreted by such different assumptions and theoretical orientations using different terminologies that one wonders whether the theorists are really thinking of the same datum” (Harbodt 1974, p. 258 f., own translation).

The proposal mentioned above to solve the under-determination problem by sim- ulation does not touch the underlying reference problem at all. It just extends the theory, adding to it its “implications”, hoping them to be more easily testable than the theory’s core theorems. The general reference between theoretical statement— be it implication or core theorem—and observed data has not changed by applying this extension: The point here is that we cannot establish a relation of uniqueness between the observed data and the theoretical statement. This applies to any segment of theorising at the centre or at the periphery of the theory on any level—a matter that cannot be improved by a cumulative strategy.

1 We owe the suggestion that simulation could be a tool to make theories more determined by data to one of the referees of Ahrweiler and Gilbert (2005).

w.jager@rug.nl

3 The Quality of Social Simulation: An Example from Research Policy Modelling 39

3.1.1.2 The Theory-Ladenness of Observations

Observations are supposed to validate theories, but in fact theories guide our ob- servations, decide on our set of observables, and prepare our interpretation of the data. Take, for example, the different concepts of two authors concerning Venetian cafés: For one, a Venetian café is a quiet place to read newspapers and relax with a good cup of coffee; for the other, a Venetian café is a lively place to meet and talk to people with a good cup of coffee. The first attribute of these different conceptions of a Venetian café is supported by one and the same observable, namely the noise level, although one author expects a low level, the other a high one. The second attribute is completely different: the first conception is supported by a high number of newspaper readers, the second by a high number of people talking. Accordingly, a “good” simulation would mean a different thing for each of the authors. A good simulation for one would be a poor simulation for the other and vice versa. Here, you can easily see the influence of theory on the observables. This example could just lead to an extensive discussion about the “nature” of a Venetian café between two authors, but the theory-ladenness of observations again leads to more serious difficulties. Our access to data is compromised by involving theory, with the con- sequence that observations are not the “bed rock elements” (Balzer et al. 1987) our theories can safely rely on. At the very base of theory is again theory. The attempt to validate our theories by “pure” theory-neutral observational concepts is mistaken from the beginning.

Balzer et al. summarise the long debate about the standard view on this issue as follows: “First, all criteria of observability proposed up to now are vulnerable to serious objections. Second, these criteria would not contribute to our task because in all advanced theories there will be no observational concepts at all—at least if we take ‘observational’ in the more philosophical sense of not involving any theory. Third, it can be shown that none of the concepts of an advanced theory can be defined in terms of observational concepts” (Balzer et al. 1987, p. 48). Not only can you not verify a theory by empirical observation, but you cannot even be certain about falsifying a theory. A theory is not validated by “observations” but by other theories (observational theories). Because of this reference to other theories, in fact a nested structure, the theory-ladenness of each observation has negative consequences for the completeness and self-sufficiency of scientific theories (cf. Carrier 1994, pp. 1–19). These problems apply equally to simulations that are just theories in process.

We can give examples of these difficulties in the area of social simulation. To compare Axelrod’s The Evolution of Cooperation (Axelrod 1984) and all the subse- quent work on iterated prisoners’ dilemmas with the “real world”, we would need to observe “real” IPDs, but this cannot be done in a theory-neutral way. The same problems arise with the growing body of work on opinion dynamics (e.g. Deffuant et al. 2000; Ben-Naim et al. 2003; Weisbuch 2004). The latter starts with some sim- ple assumptions about how agents’ opinions affect the opinions of other agents, and shows under which circumstances the result is a consensus, polarisation, or fragmen- tation. However, how could these results be validated against observations without involving again a considerable amount of theory?

w.jager@rug.nl

40 P. Ahrweiler and N. Gilbert

Important features of the target might not be observable at all. We cannot, for example, observe learning. We can just use some indicators to measure the conse- quences of learning and assume that learning has taken place. In science simulations, the lack of observability of significant features is one of the prime motivations for carrying out a simulation in the first place.

There are also more technical problems. Validity tests should be “exercised over a full range of inputs and the outputs are observed for correctness” (Cole 2000, p. 23). However, the possibility of such testing is rejected: “real life systems have too many inputs, resulting in a combinatorial explosion of test cases”. Therefore, simulations have “too many inputs/outputs to be able to test strictly” (Cole 2000, p. 23).

While this point does not refute the standard view in principle but only emphasises difficulties in execution, the former arguments reveal problems arising from the logic of validity assessment. We can try to marginalise, neglect, or even deny these problems, but this will disclose our position as mere “believers” of the standard view.

3.1.2 The Constructivist View

Validating a simulation against empirical data is not about comparing “the real world” and the simulation output; it is comparing what you observe as the real world with what you observe as the output. Both are constructions of an observer and his/her views concerning relevant agents and their attributes. Constructing reality and simu- lation are just two ways of an observer seeing the world. The issue of object formation is not normally considered by computer scientists relying on the standard view: data is “organized by a human programmer who appropriately fits them into the chosen representational structure. Usually, researchers use their prior knowledge of the na- ture of the problem to hand-code a representation of the data into a near-optimal form. Only after all this hand-coding is completed is the representation allowed to be manipulated by the machine. The problem of representation-formation [. . .] is ignored” (Chalmers et al. 1995, p. 173).

However, what happens if we question the possibility of validating a simulation by comparing it with empirical data from the “real world”? We need to refer to the modellers/observers in order to get at their different constructions. The constructivists reject the possibility of evaluation because there is no common “reality” we might refer to. This observer-oriented opponent of the realist view is a nightmare to most scientists: “Where anything goes, freedom of thought begins. And this freedom of thought consists of all people blabbering around and everybody is right as long as he does not refer to truth. Because truth is divisible like the coat of Saint Martin; everybody gets a piece of it and everybody has a nice feeling” (Droste 1994, p. 50).

Clearly, we can put some central thoughts from this view much more carefully: “In dealing with experience, in trying to explain and control it, we accept as legitimate and appropriate to experiment with different conceptual settings, to combine the flow of experience to different ‘objects”’ (Gellner 1990, p. 75).

w.jager@rug.nl

3 The Quality of Social Simulation: An Example from Research Policy Modelling 41

However, this still leads to highly questionable consequences: There seems to be no way to distinguish between different constructions/simulations in terms of “truth”, “objectivity”, “validity”, etc. Science is going coffeehouse: Everything is just construction, rhetoric, and arbitrary talk. Can we so easily dismiss the possibility of evaluation?

3.1.3 The User Community View

We take refuge at the place we started from: What happens if we go back to the Venetian café simulation and ask for an evaluation of its performance? It is probably the case that most customers in the Guildford Caffè Nero have never been to an Italian café. Nevertheless, they manage to “evaluate” its performance—against their concept of an Italian café that is not inspired by any “real” data. However, there is something “real” in this evaluation, namely the customers, their constructions, and a “something” out there, which everybody refers to, relying on some sort of shared meaning and having a “real” discussion about it. The philosopher Searle shows in his work on the Construction of Social Reality (Searle 1997) how conventions are “real”: They are not deficient for the support of a relativistic approach because they are constructed.

Consensus about the “reality observed by us” is generated by an interaction pro- cess that must itself be considered real. At the base of the constructivist view is a strong reference to reality, that is, conventions and expectations that are socially cre- ated and enforced. When evaluating the Caffè Nero simulation, we can refer to the expert community (customers, owners) who use the simulation to get from it what they would expect to get from the target. A good simulation for them would satisfy the customers who want to have the “Venetian feeling” and would satisfy the owners who want to get the “Venetian profit”.

For science, equally, the foundation of every validity discussion is the ordinary everyday interaction that creates an area of shared meanings and expectations. This area takes the place left open by the under-determination of theories and the theo- reticity problem of the standard view.2 Our view comes close to that of empirical epistemology, which points out that the criteria for quality assessment “do not come from some a priori standard but rest on the description of the way research is actually conducted” (Kértesz 1993, p. 32).

2 Thomas Nickles claims new work opportunities for sociology at this point: “the job of phi- losophy is simply to lay out the necessary logico-methodological connections against which the under-determination of scientific claims may be seen; in other words, to reveal the necessity of so- ciological analysis. Philosophy reveals the depths of the under-determination problem, which has always been the central problem of methodology, but is powerless to do anything about it. Under- determination now becomes the province of sociologists, who see the limits of under-determination as the bounds of sociology. Sociology will furnish the contingent connections, the relations, which a priori philosophy cannot” (Nickles 1989, p. 234 f.).

w.jager@rug.nl

42 P. Ahrweiler and N. Gilbert

If the target for a social science simulation is itself a construction, then the simu- lation is a second-order construction. In order to evaluate the simulation, we can rely on the ordinary (but sophisticated) institutions of (social) science and their practice. The actual evaluation of science comes from answers to questions such as: Do others accept the results as being coherent with existing knowledge? Do other scientists use it to support their work? Do other scientists use it to inspire their own investigations?

An example of such validity discourse in the area of social simulation is the history of the tipping model first proposed by Schelling, and now rather well known in the social simulation community. The Schelling model purports to demonstrate the reasons for the persistence of urban residential segregation in the USA and elsewhere. It consists of a grid of square cells, on which are placed agents, each either black or white. The agents have a “tolerance” for the number of agents of the other colour in the surrounding eight cells that they are content to have around them. If there are “too many” agents of the other colour, the unhappy agents move to other cells until they find a context in which there are a tolerable number of other-coloured agents. Starting with a random distribution, even with high levels of tolerance, the agents will still congregate into clusters of agents of the same colour. The point Schelling and others have taken from this model is that residential segregation will form and persist even when agents are rather tolerant.

The obvious place to undertake a realist validation of this model is a US city. One could collect data about residential mobility and, perhaps, on “tolerance”. However, the exercise is harder than it looks. Even US city blocks are not all regular and square, so the real city does not look anything like the usual model grid. Residents move into the city from outside, migrate to other cities, are born and die, so the tidy picture of mobility in the model is far from the messy reality. Asking residents how many people of the other colour they would be tolerant of is also an exercise fraught with difficulty: the question is hypothetical and abstract, and answers are likely to be biased by social desirability considerations. Notwithstanding these practical methodological difficulties, some attempts have been made to verify the model. The results have not provided much support. For instance, Benenson (2005) analysed residential distribution for nine Israeli cities using census data and demonstrated that whatever the variable tested—family income, number of children, education level— there was a great deal of ethnic and economic heterogeneity within neighbourhoods, contrary to the model’s predictions.

This apparent lack of empirical support has not, however, dimmed the fame of the model. The difficulty of obtaining reliable data provides a ready answer to doubts about whether the model is “really” a good representation of urban segregation dy- namics. Another response has been to elaborate the model at the theoretical level. For instance, Bruch (2005) demonstrates that clustering only emerges in Schelling’s model for discontinuous functional forms for residents’ opinions, while data from surveys suggest that people’s actual decision functions for race are continuous. She shows that using income instead of race as the sorting factor also does not lead to clustering, but if it is assumed that both race and income are significant, segregation appears. Thus, the model continues to be influential, although it has little or no em- pirical support, because it remains a fruitful source for theorising and for developing

w.jager@rug.nl