2.1 BDA Adoption Challenges

EA should help to overcome BDA adoption challenges and understanding these challenges requires to understand the roles and capabilities of EA. Traditionally, organizations use well-structured data that is collected for a certain purpose and stored, often in relational databases. Basic queries can be executed based on this data. Over time, the number of databases, the variety of formats and the amount of data has grown into what is now called BDA [14]. The use of BDA enables organizations to go beyond mere utilization of each database and data set in isolation. Data sets are now combined or integrated to achieve new insights.

While many studies have enhanced our understanding of the BDA phenomenon, most emphasize the technical dimension of BDA and understate the importance of the organizational dimension, which is equally important to reap the benefits of BDA [28]. Unlike the more tangible BDA technical capabilities, those in the organizational dimension are often intangible and concern the collaboration and strategy of the organization [13]. Organizational capabilities address whether organizations possess the required capacities to use BDA and to create value from it for the organization [35]. If these organizational capabilities are not sufficient, the benefits generated from the use of BDA will be limited [35]. In the current climate, BDA is expected to create value for organizations. However, realizing such value is much more difficult, and many challenges will be encountered during the adoption of BDA technologies and the implementation of BDA application [26], [59]. For example, based on a systematic literature review, Sivarajah, et al. [59] identified 18 BDA challenges, classifying them into three main categories: data, process and management. Data challenges relate to the characteristics of the data itself, for example data volume, variety, velocity, veracity, volatility, quality, discovery and dogmatism. Process challenges relate to a series of how questions: how to capture data, how to integrate data, how to transform data, how to select the right model for analysis and how to provide the results. Management challenges relate to the management strategy or policy constraints in BDA adoption, for example privacy, security, governance and ethical aspects. Sivarajah, et al. [59] found that different sectors and organizations will face different types of challenges and that there is a need for more research using in-depth case studies in organizations in different sectors.

In the literature, only a limited number of articles have investigated the management challenges of BDA adoption. The remainder mainly focus on the technologies or benefits of BDA, the introduction of a certain BDA application in a sector, or discuss the data and process challenges. Based on our literature survey, Table 1 presents the main management challenges of BDA adoption to which organizations need to respond strategically.

Table 1: Typical management challenges of BDA adoption for large organizations 

*Many studies mentioned privacy as a BDA challenge. To avoid of confusion, only those focusing on privacy issues from an organizational or strategic perspective are listed in the table.

Many systems are fragmented and have developed without considering the need to exchange information. Siloed applications are often encountered in many large organizations, or when different organizations start to work with each other. Siloed processes can originate from legal requirements, in which personal data cannot be shared due to data protection laws, or from a rigid institutional setup and routine that pose major obstacles [25]. Organizations such as government agencies or departments are often reluctant to share data that is considered to be proprietary to them. Furthermore, isolated IT systems complicate the integration of complementary data within the organization [34]. This results in a need to break down the silos or to integrate data and applications. Furthermore, addressing the silo problems consequently requires the organization to balance among integration and privacy protection, compliance and/or accountability.

The requirements for data protection found in regulatory frameworks such as the General Data Protection Regulation (GDPR) determine the way in which BDA can be used by organizations, with privacy laws restricting the collection, sharing or utilization of personal data [25]. At the same time, additional data from citizens, the neighborhood or the community is needed to offer personalized services or better governance, for example in relation to traffic control, energy consumption, disease prevention, public security, personalized services and so on. Ultimately, there is a need to find a balance between privacy protection and other values or general interests [19].

Organizations need management capabilities to handle BDA and to realize its expected outcomes and may address this by employing staff, creating agencies and building new operational business processes. Management capabilities relates to the organizational capacity to make use of BD, and it is often assessed in terms of BDA readiness [25], [35]. The concept of BDA readiness focuses on the organizational alignment and capabilities in connection to BDA. From an intra-organizational perspective, alignment refers to whether BDA use is a good fit with the organization’s structure and main activities. From an inter-organizational perspective, this is also referred to as engagement capacity, which captures the ability of the organization to digitally interface with external groups and organizations to make use of BDA [70].

In addition to data protection, organizations must address security and compliance requirements when using data. However, unlocking the full potential of BDA first requires the authority to develop knowledge and skills and to allow innovative use of data [40]. In other words, to facilitate BDA adoption and drive organizational change, managers might first need to legitimate their actions [41]. The use of BDA can improve services, but can also undermine trust in organizations, as customers question the legitimacy of providing or allowing access to sensitive data [44]. These kinds of concerns might be addressed at the legislative level to create an appropriate balance. However, for public organizations, it is more important to ensure the accountability of their business processes and decision-making. This might entail that they ensure the organization is able to check the compliance of their service provision and decision-making processes with the changing legal environment.

The above management challenges for organizations indicate the special concerns about BDA adoption. At the same time, however, limited knowledge about BDA adoption from an organizational perspective has been reported in the literature. Organizations are postponing decisions on BDA adoption because they are uncertain about how these challenges could be addressed to take advantage of BDA [35]. It is, therefore, desirable to investigate those challenges and how they might be overcome with the help of EA.

2.2 EA Roles and Capabilities for BDA Adoption

Along with the adoption of new information technology, the complexity of corporate IT environments has been growing steadily. The growth of IT complexity often leads to various problems, such as higher operational risks and costs for maintenance or operation, and insufficient flexibility with regard to changes in business strategy and processes [56]. In practice, EA provides big-picture planning and understanding of the interdependences between various IT components, which is of critical importance in solving these problems [60], [62]. EA provides a range of capabilities to deal with technology aimed at improving organizational performance [53]. EA captures a range of aspects, including business processes, data models, applications and technical infrastructure [64]. EA can have various roles in organizations for managing business and IT [27}. Table 2 Table 3 summarizes the roles of EA as discussed in the literature. Usually, EA consists of descriptive and prescriptive models to understand the current situation and what the desired situation should look like [31]. The architecture models should help to improve the relevant parts of the organization (i.e. the business process, data, application and infrastructure) and take into account the relationships as they are depicted by the architecture. In this way, a coherent picture is created. This approach assumes that the situation desired can be blueprinted to some extent and that the relationship with the current situation is clear. Another view of EA considers it a process in which the focus is on managing the improvement process supported by architectural instruments [61].

Table 2: The roles of EA in organizations 

Despite its significance, there is limited work on the role of EA in supporting new technology adoption. The exceptions include studies investigating EA’s effect on the organizational adoption of service-oriented architecture (SOA) [5] and those pointing to EA’s importance in cloud computing adoption [21]. These studies are focused on technology that are primarily changing the IT functions, however, BDA is used by the businesses directly to create value for the organizations. This requires complex interactions between the IT and Business parts of an organization. In addition, some earlier work has investigated the use of EA in technology adoption in a more general manner; for example, looking at the use of EA artefacts in IT acquisition management by providing a description of the organization and its requirements for vendors [10]. However, existing studies have often focused on the benefits of EA rather than an analysis of how EA can be used to adopt new technology. For example, in one study of IT acquisition, EA artefacts were purely regarded as background material [46]. The limited findings in the literature reveal the knowledge gap in understanding the role of EA in new technology adoption, and especially in the adoption of BDA.

Management practice and research is increasingly incorporating new technology adoption into EA-related issues [71]. EA can be used to develop a common understanding or interpretation of existing and new technologies, and as such could be used as a tool by organizations to understand the potential and feasibility of new technologies [8]. Shanks, et al. [57] suggest that EA will enable organizations to adopt new technologies more successfully and consequently lead to quicker response to changes in customer demand by offering relevant new products and services and better strategic alignment between business and IT in contrast to competitors. However, EA-based benefit realization for organization is a complex process and is influenced by EA maturity and capability [12]. EA is still a rather young and immature discipline, for which no common practice has yet emerged [56]. In practice, different organizations have often employed different approaches, depending on their strategies and EA capability.

EA should create the necessary capabilities for organizations to adopt BDA technology. According to Winter [72], an organizational capability is a high level routine (or collection of routines) that, together with its implementing input flows, confers upon an organization’s management a set of decision options for producing significant outputs of a particular type (p. 991). In contrast with ordinary (or operational) capabilities that focus on ordinary routines, dynamic capabilities focus on the organizational strategic routines [20] and govern the rate of the change of ordinary organizational capabilities [16]. Teece, et al. [63] define dynamic capabilities as the ability to integrate, build, and reconfigure internal and external competences to address rapidly changing environments (p. 516). Dynamic capabilities are needed to integrate BDA in the organization and reconfiguring is needed to learn from the experiences. From this lens, EA is an enabler of dynamic capabilities of the organization. The better the EA function is developed the better the capabilities of an organization to adopt new technology. While EA is expected to have an impact and evolve continuously in the BDA movement, how EA can be used to facilitate effective applications of BDA is still an open question [52]. There is a void in the literature addressing the roles and capabilities for adopting BD. This paper attempts to explore this question by addressing the roles and dynamic capabilities of EA in BDA adoption.

5.1 BDA Challenges and the Use of EA

The case study presents a change in BDA adoption strategy, from an internally driven to an externally driven approach over time. Both strategies needed EA to support BDA adoption and development. By examining the type of management challenges, listed in Table 1 above, we found that the change in strategy introduced different management challenges to the organization. As different challenges were encountered, the use of EA shifted, along with the change in BDA adoption strategy. The bases of both strategies, the BDA challenges in the organization and an overview of the use of EA are shown in Table 4. The BDA challenges are summarized in a map with the management challenges of BDA adoption from the literature (Table 1).

Table 4: BDA Adoption challenges and the use of EA 

In the internally driven strategy, the idea was to facilitate bottom-up initiatives and to gradually incorporate the results of the initiatives into the EA. The bottom-up initiatives were expected to develop software components and introduce new processes and procedures for dealing with BD. The departments of the tax organization employed EA to understand the existing IT landscape and how the existing systems that they used connected to the infrastructure. This allowed them to review implementation feasibility of the proposed BDA application before use. In addition, EA indicated the existing data sources that could be used by the proposed BDA application, while the interoperability principles determined the system requirements. To some extent, EA also enabled the evaluation of the proposed BDA application in a cost-benefit analysis.

At first, the tax organization expected this strategy to allow for the integration of BDA applications within the existing IT landscape and thus present a highly cost-beneficial solution through the reuse of existing resources. However, the initiatives were found to be overly diverse and did not create cross-departmental applications. They were proven to be insufficiently advanced to provide undisputed input into the EA. Moreover, the existing departments did not have the capabilities to make use of this advanced technology. In addition, it was unclear how the existing systems should be updated and how data governance should be dealt with. Overcoming these challenges was very time-consuming and the EA group and ARB found themselves overworked.

The literature suggests that EA should be used as a blueprint to specify a certain implementation or provide guidelines for implementation on a higher level [4], [46]. However, the assumption that the desired situation can be blueprinted to integrate BDA was not found valid in our case. Only a long-term vision can be developed, but there are so many complexities and changes that a detailed description of the future situation is not possible. Complexities were found at different levels, which proved to make it too complicated to develop a blueprint in which BDA could be integrated into the existing structure. The infrastructure was not able to handle the large amount of data, data with various formats proved not to be interoperable and people did not have the skills and expertise for BDA analysis. These issues needed to be resolved and understood before a blueprint could be made to integrate BDA use into the current infrastructure.

The above challenges resulted in difficulty in integrating BDA into the current IT landscape. This resulted in the creation of a separate department, which operated independently from the existing organization. BDA experts were hired and used to create this department. New technology was used, which was different from the conventional technologies. This effort came with the creation of added value applications, such as dynamic monitoring of the economic activities of people and organizations who did not pay their taxes. Based on this strategy, EA was used as a collection of existing standards and data sources. This allowed the new BDA principles and standards to be considered alongside the existing ones. The reusability of BDA applications was also considered by using EA as the input for deriving requirements from different departments.

Separation from the current IT landscape made the process of BDA development faster than the first strategy. However, this also resulted in the creation of a separate domain, which could not be integrated with the existing IT landscape. This neglect of the existing systems created another kind of silo, in which the lack of collaboration among people with different roles and across processes hindered the collection of data. Furthermore, as most of the newly hired people were not familiar with the tax organization, the new systems did not adhere to the required norms and public values. In particular, the storage of all the data, with the ability to link this to person, was found to be in violation of the data protection act. There was also friction between the departments in charge of the collection of the data and the external department conducting data analytics.

In the third phase, the importance of systems integration and data governance has been realized. EA is expected to facilitate the integration of processes and systems and enable data management across departments. In a comparison with the first phase, the organization encountered the similar issues regarding the adaptation of infrastructure and data interoperability. This implies the need of comprehensive capabilities beyond the choice of BDA adoption and implementation strategies.

5.2 Developing Dynamic Capabilities

Organizations are often uncertain about what dynamic capabilities are required for BDA adoption and implementation before they start their BDA initiatives [35]. BDA requires new capabilities that are not readily available in the organization. Developing new BDA applications requires that staff should be able to program, analyze the data, have knowledge of statistics and be a good communicator. These capabilities can either be developed in-house through training and experimentation or can be bought on the market and subsequently internalized. Knowledge transfer and information flows from both internal and external sources play an important role in the creation of dynamic capabilities.

One way the tax organization could overcome the challenges encountered in BDA adoption and implementation was to develop a new set of capabilities that it did not previously possess. These capabilities (see Table 5) range from the strategic to the technical. Lacking these capabilities makes it difficult to realize the benefits of BD. In particular, the many changes that were needed could not be dealt with by the organization. Multiple interviewers indicated that the organization did not have sufficient dynamic capability to deal with all the changes at the same time.

Table 5: An overview of dynamic capabilities needed for BDA adoption and implementation in the tax organization 

To overcome the challenges of flexibility and interoperability, the organization is required to fundamentally improve its infrastructure and software technology. In the case study, the current EA documents informed BDA developers about the location and content of data, but this was found to provide limited help in integrating BDA into the existing infrastructure and software. The findings suggest that current infrastructures might not be suitable for integrating BDA, and substantial changes are needed. The main question for the organization is therefore not how to use current EA documents to manage the integration of BDA, but how to redesign the infrastructure to make it suitable for BDA implementation, as well as develop a migration plan for it by making use of EA.

Standardization is also important for enabling efficient data exchange and reuse, and consequently facilitates interoperability. Various data formats were documented by EA files as input for defining BDA standards at the organizational level. However, standardization is still quite a difficult job and has uncertainties, as it is not only affected by the current use of data within an organization but also by changes in industrial or national standards, and the possible use of BDA in the future. Moreover, even when infrastructure and standardization are adjusted for the use of BD, people need sufficient BDA skills to be able to realize the benefits.

Furthermore, to avoid fragmentations, the BDA value-creation process should be negotiated and designed in a collaborative network, allowing further access to and usage of data. Data governance that ensures the quality of data and the appropriate use of it is important to protect privacy and ensure legitimacy, thereby maintaining a social structure suitable for BDA development. This goes beyond the current use of EA for documenting who is doing what in a business process. The new capabilities for collaboration will allow the organization to redesign the business processes across different departments from a value-creation perspective, rather than simply connecting existing processes.

Finally, leadership support is essential in BDA implementation to ensure authority. Public organizations often create institutional structures through the division of their business and services. The use of BDA often does not fall to a single department, but has an impact on a larger scale, creating new insight for the whole organization. Top management could use EA as input to redesign the organizational structure and make appropriate adjustments to existing institutionalized patterns to provide authority for BDA implementation. This means top management is able to understand and can make decisions based on freeing up resources, and support the use of BDA in a global and long-term perspective.

Possessing the above capabilities allows an organization to reconsider the role of EA in BDA adoption and implementation. The case study implies that given sufficient capabilities, EA could play a more important role in BDA adoption and implementation. This will be discussed in the following section.

5.3 Roles of EA in BDA Adoption and Implementation

The investigation of different BDA adoption strategies gives us an insight into the effect of using EA in BDA adoption. However, this only reflects a narrow view on EA functions, whereas EA had an even greater role in BDA development. To understand the potential roles of EA in BDA adoption and implementation, we confronted the roles of EA as found in the literature and the findings of this case study. EA roles were used in various ways, as shown in Table 6. The possible roles of EA as found in the literature are outlined in the left-hand column. The roles found in our case study are listed in the right-hand column. The table clearly shows that multiple roles of EA were of importance. Only client orientation and decision-making support were not found to be important in the case study: although BDA helps to better understand clients, EA was not used to enable this; and the decision to use BDA had already been made without the support of EA. Furthermore, the roles of EA changes over time and needs to be appropriate. This suggests that EA should not be viewed as a one size that fits, but should be customized and the roles are context dependent.

Table 6: Overview of the roles of EA in BDA adoption and implementation 

The elements presented in Table 6 allow a more comprehensive understanding of the potential effect of EA in BDA adoption and implementation. EA was not found to be important for client orientation, as BDA efforts in the tax organization were more focused on the back end. Moreover, as mentioned above, EA did not play a role in decision-making support as the top management made the decision to implement BDA without fully realizing the strategic benefit of EA. This table enables a broader view of the roles of EA and complements the narrow view that only focuses on management challenges for organizations. With an awareness of the functions of EA, as identified in our case study, whether the contribution of a certain EA function can be maximized will depend on the way EA is utilized. The internally driven and externally driven approaches in this case study reflect two roles of EA: EA as a blueprint and EA as a process.

5.4 Lessons Learned

The BDA efforts were largely pushed by the board, which gave limited space to the EA group to review and suggest BDA strategy. The BDA phenomenon was surrounded by uncertainties, which made it difficult to use the architecture models to determine its impact. Instead, an experimenting and learning by doing approach was adopted. This EA as a process approach enabled the creation of some BDA applications by departments and demonstrated the possibilities of BD. However, this resulted in higher costs, due to the fragmentation of initiatives and control and maintenance problems, and it was difficult to manage software and data quality. One interviewee commented that wrong data could spread like an oil slick in the organization. Data stewardship and ability to manage data is needed.

Although the architecture models were little used to understand the implications, the models did provide an overview of the data sources. One interviewee emphasized the importance of EA: Data should be given only once to the government. EA should ensure that the departments reuse the data. The expectation is that once it becomes clear what BDA is and how it can be used, there will be a larger role for the architecture models to work as a blueprint. In particular, the models can be used for impact analysis and prescribing the desired infrastructure. Given the many uncertainties and lack of understanding of what BDA should look like and how the existing landscape should be adapted, EA as a blueprint was little used for BDA adoption in our case study.

The existing architecture was not ready for BDA. A move from built to last to built to change was necessary according to the interviewees. System quality was not sufficient, as data could not be easily extracted, systems were not sufficiently flexible and new functionality could not be integrated into the existing architecture. The adoption and implementation of BDA requires that the existing systems are prepared for this. The manager of the EA group stated that: EA should enable the reconfiguring of the applications and data for different purposes. In the case study, however, a new system was created at each stage. As such, an incremental strategy was followed. This approach emphasizes the roles of EA in dealing with complexity, integrating new systems into the IT landscape, facilitating reuse and communication.

Defining the starting point of BDA initiatives is a difficult endeavor. The strategy of attempting to integrate BDA into the current architecture proved not to work and, therefore, another approach was taken. Efforts were made independently of the organization, with the idea of integrating the new applications at a later stage. Nevertheless, the data was to be collected from the existing organization and an agile approach was taken by involving policymakers, administrators, domain (tax), and BDA quality and analytics experts. These teams developed the ideas for the applications, collected the data and processed the data to create new insights. These multidisciplinary teams included people from the current department and the newly founded BDA department.

The organization under study facilitated bottom-up initiatives. Although this stimulated the development of new applications and the use of BD, it also resulted in fragmentation and encountered many problems, generating limited value creation from BD. In particular, the departments only analyzed their own data, without looking at the potential to use data from other departments, which limited the value that could be created. Although the bottom-up initiatives were instrumental to the acceptance of the use of BD, they created difficulties for the distinct BDA department. They were sometimes seen as unnecessary and used different technologies. In the long run, this created integration problems. Developing an incremental migration plan was found to be key to the adoption and implementation of BDA.

Both applying EA as a blueprint and a process can have benefits for an organization in the adoption and implementation of BDA. A process approach allows for quick gains, but has less ability to deal with uncertainty surrounding BDA development. Adopting BDA is challenging, as substantial changes are needed. A blueprint approach should deal with the uncertainties and complexities by ensuring a configurable IT landscape, and providing an incremental approach to adapting the infrastructure step by step before the benefits of BDA can be obtained. However, the use of EA as a blueprint requires the organization to have sufficient dynamic capabilities to enable BDA implementation.