Data Warehousing Fundamentals For It Professionals
OUR TAKE: This title was specifically written for professionals responsible for designing, implementing, or maintaining data warehousing systems. It is also relevant for those working in research and information management.
This practical Second Edition highlights the areas of data warehousing and business intelligence where high-impact technological progress has been made. Discussions on developments include data marts, real-time information delivery, data visualization, requirements gathering methods, multi-tier architecture, OLAP applications, Web clickstream analysis, data warehouse appliances, and data mining techniques. The book also contains review questions and exercises for each chapter, and is appropriate for self-study or classroom work.
Decentralized Execution For Team
This model is also built for business owners who manage and maintain their master data. However, in this model, data is used and shared by several employees across different teams. This way, if your business has several offices or stores, you can ensure information is categorized and distributed to every person on your team. Check out how this model works using the image below.
Data Management Vs Data Governance: An Introduction
If theres anything thats defining successful businesses today, its the successful understanding, use, and strategy of a companys data. Understanding your data and determining how to implement it brings up a whole range of questions, from both users and stakeholders:
- How is the data stored?
- How do we know its timely and accurate?
- Can we trust it?
- What is the best data for my problem?
Answers to these questions arent easy, but a couple fields provide ways to organize and solve them: data management and data governance. Though these terms are often used interchangeably, they are entirely different programs. In this article, were clearing up any confusion about data management and data governance.
Don’t Miss: Government Grants For College Students
Data Governance In Business Intelligence And Analytics
Every business intelligence or analytics initiative needs to have the foundation of accurate, well-managed data that comes from a robust data governance program
Many corporations experience significant business benefits using business intelligence and analytics. Users report gains in market competitiveness and data / information management .
How Do You Set Up Good Data Governance And Privacy Model
Several bricks are needed to enforce data management
Before even talking about data governance, a company needs the basis: a good infrastructure to begin with. Based on business needs and the companys data maturity, the nature of the data architecture can change a lot. Regarding storage, do you go for: on-premise or cloud? data warehouse or data lake? Regarding modeling: Spark or DBT? in a data warehouse or in a BI tool? real-time or batch? Regarding visualization: do you allow anyone to build dashboards or data teams only? etc.
The first level of any data governance strategy is making sure relevant people can find the relevant datasets to do their analysis or build their AI model. If you dont implement this step, companies end up with a lot of questions on Slack and useless meetings with the engineering teams. The company ends up with a lot of duplicate tables, analyses, and dashboards. It takes valuable time to engineering resources that are needed to perform the next steps.
Once you can efficiently find the data. You need to understand it quickly in order to assess if it is going to be useful. For example, you are looking at a dataset called active_users_revenue_2021. There is a column payment. Is this column in or $? Has it been refreshed this morning, last week, or last year? Does it contain all the data on active users or just the ones in Europe? If I remove a column, will this break important dashboards for the marketing or finance team? etc.
Also Check: Government Watch List Free Search
How To Implement Data Governance Using Best Practices
Despite the challenges above, implementing effective data governance can be smooth and efficient. Here are some of the key best practices you should follow.
Data Management Vs Data Governance
Data management is simply defined as the implementation of tools, processes as well as architectures designed to achieve your organizations objectives. On the other hand, Data governance can be defined as the management of how data is accessed and handled in a data management strategy, including authentication and access granted to users of the data and compliance procedures.
Simply put, data governance seeks to establish policies and procedures of handling data, while data management seeks to enact these policies and procedures to make meaningful use of that data for onward processing and organizational decision-making.
Let us compare these two further by defining each.
What is data management?
Data management is the development and thereafter the implementation of structures, policies, and procedures to manage the full data lifecycle in an organization. These policies and procedures are critical in an organization to help in analyzing complex and big data. Data in modern times is treated as the most important asset in any organization, and therefore, it needs to be managed as such.
In the 2019 State of Data Management report, data governance ranks in one of the top 5 strategic initiatives undertaken by global organizations in 2019.
What are some of the elements of data management?
Data preparation
Data pipelines
These are channels used to automatically transmit data from a system to another.
Data governance
Data catalogs
Data warehouses
Data extract, transform, load
Recommended Reading: Us Government Mint Gold Coins
Roles And Responsibilities On A Data Governance Team
Some organizations have a dedicated data governance team, while others have employees assume the additional responsibilities in addition to their normal duties. Theres a common misconception that data governance is only an IT job. The reality is, while IT teams are responsible for providing solutions and developing infrastructure services, other team members are just as vital for example, they provide guidance on data governance policies and rules.
The key data governance roles that need to be filled include:
- Data steward This is an operational duty that focuses on implementing and coordinating policies and procedures. Data stewards manage corporate data projects, make data-related decisions, issue recommendations and develop relevant policies.
- Data governance council This team is responsible for setting up the data governance program, measuring success and gathering metrics.
- Data stakeholders These are the people who own and use specific data assets. They usually include individuals and teams in the human resources, IT, risk management, compliance and legal departments. Their insights and needs should be considered in decisions about policies, procedures, business rules and technology approaches.
Grow Up Kid: The Maturity Model
Measuring your organization up against a data governance maturity model can be a very useful element in making the roadmap and communicating the as-is and to-be part of the data governance initiative and the context for deploying a data governance framework.
One example of such a maturity model is the Enterprise Information Management maturity model from Gartner, the analyst firm:
Figure 2.
Most organizations will, before embarking on a data governance program, find themselves in the lower phases of such a model.
Phase 0 Unaware: This might be in the unaware phase, which often will mean that you may be more or less alone in your organization with your ideas about how data governance can enable better business outcomes. In that phase you might have a vision for what is required but need to focus on much humbler things as convincing the right people in the business and IT on smaller goals around awareness and small wins.
Phase 1 Aware: In the aware phase where lack of ownership and sponsorship is recognized and the need for policies and standards is acknowledged there is room for launching a tailored data governance framework addressing obvious pain points within your organization.
Phases 4 and 5 Managed & Effective: By reaching the managed and effective phases your data governance framework will be an integrated part of doing business.
Read Also: Federal Government Dental And Vision Insurance
Sap Master Data Governance
Best for: Enterprise companies
SAP Master Data Governance allows you to govern data in two ways: one, you can decentrally own and consolidate it across your enterprise system landscape or, two, you can centrally create, change, and distribute master data across that landscape. With this tool, you can consolidate databases, make changes in bulk, analyze the impact of your data governance processes, and define, validate, and monitor rules for data quality.
How Do Data Governance And Data Management Work Together
Data management without governance is like constructing something without a blueprint. Meanwhile, governance without management is just documentation. Its crucial for data governance and data management to work in tandem so that you can extract value from your data.
Here are some examples of how that would look:
Also Check: Local Federal Government Credit Union Member Connect
What Is A Data Governance Framework
Venn diagram showing how a data governance framework creates a single set of rules and processes for collecting, storing and using data.
Data governance requires your organization to understand and take stock of your data. Which regulatory and legal requirements apply to your data? Which business best practices are most appropriate for your organization?
Once you have that understanding, you need to establish rules and adopt automated and human processes to enforce those rules. The drivers of data governance are usually regulatory and legal requirements, but the organization determines which rules to include.
Governance often dictates policies such as storage for certain types of data. It also codifies data protection methods, such as encryption or password strength. Data governance can dictate how to back up data, decide who has access, and sets guidelines for when you should destroy archived data. You can also set governance objectives around issues such as data quality or silos that isolate certain data.
You often hear about data governance frameworks. A data governance framework consists of the data strategy policies that impact everyone in the organization:
- Processes and procedures
If data governance is the what, then a data governance framework is the how, and alignment to your strategy, the why.
What Are The Key Differences Between Data Governance And Data Management
Now lets consider data governance vs data management. Data governance is a broad set of policies, implemented across the organization. The concept of data management is narrower, focused on executing the specific processes that support the data governance policy.
In other words, when it comes to data governance vs data management, data management is the execution and data governance is the guidance that informs the execution.
Recommended Reading: How To Get Government Security Clearance
Data Protection And Data Privacy
The increasing awareness around data protection and data privacy, for example, manifested by the European Union General Data Protection Regulation have a strong impact on data governance.
Terms such as data protection by default and data privacy by default must be baked into our data policies and data standards not least when dealing with data domains such as employee data, customer data, vendor data and other party master data.
As a data controller, you must have full oversight over where your data is stored, who is updating the data and who is accessing the data for what purposes. You must know when you handle personally identifiable information and do that for legitimate purposes in the given geography both in production environments and in test and development environments.
Having well-enforced rules for the deletion of data is a must too in the compliance era.
Key Data Governance Pillars
Data governance programs are underpinned by several other facets of the overall data management process. Most notably, these facets include the following:
Data governance is also related to information governance, which focuses more broadly on how information is used overall in an organization. At a high level, data governance can be viewed as a component of information governance, but they’re generally considered to be separate disciplines with similar aims. Get an explanation of how data and information governance differ in an article by Lawton.
Recommended Reading: How Can I Get Government Assistance For Housing
Data Governance Is A Trending Topic Mainly Focused On Bi And Data Warehousing And Mainly Driven By Compliance
Only 4 percent of participants regard data strategy and data governance as an inconceivable approach for their business now and in the future. A majority focus their current governance activities on business intelligence and the data warehouse. Best-in-class companies have realized that it is important to cover all data environments and establish a feedback loop from data usage in BI and analytics to drive data improvements. While compliance is the major driver for data governance, it bears the risk of reducing it to a very restrictive procedure.
Data quality is the top challenge when it comes to using data, closely followed by organizational issues. Inadequate data quality remains the foremost challenge users face when using data. This has been shown repeatedly by market research and data-centered projects for many years.
Yet it appears that the reasons for the apparent inability of businesses to cope with this problem in order to achieve continuous improvement are largely organizational. In spite of pressing data quality problems, there is a lack of acceptance and priority for data governance at executive level and in lines of business.
Benefits Of Data Governance
Every organization should have some form of data governance to prevent sensitive information from getting into the wrong hands. And if your company is large or in an industry subject to regulation such as healthcare or banking, data governance is critical.
The right practices, processes, and tools for handling and using your data ensure data security, availability, and quality so everyone within the organization can use the data to fix business problems and discover opportunities. Additional benefits:
Regulatory Compliance
You may be considering your data governance strategy as part of your need to comply with regulatory policies such as the European Unions General Data Protection Regulation or the United States Health Insurance Portability and Accountability Act . These and other regulations necessitate that you trace your data from source to retirement, identify who has access, and know how and where it is used. Effective data governance ensures that data wont fall into the wrong hands or be improperly removed.
Data Security
Data breaches, including the theft of information and inappropriate access to data, are an all-too-common occurrence. As with regulatory compliance, data security depends on traceability knowing where the data comes from, where it is located, who has access to it, how it is used, and how to delete it upon request. Effective data governance prevents data leaks and misuse, thereby protecting your organizations reputation and revenue.
Don’t Miss: What Is The Us Government Debt
Data Governance Is Not Optional
Organizations today have incredible amounts of data about customers, clients, suppliers, patients, employees, and more. When this information is properly used to better understand the market and your target audience, an organization will be more successful. The same data governance will also ensure this data is trusted, well-documented, and easy to find and access within your organization, and that it is kept secure, compliant, and confidential.
Make certain that your organization is positioned to maximize data governance investments and minimize risk of data breaches. Take a look at our data governance solutions when youre ready to get started.
Outline Your Supply Chain Of Data
Each organization has its own data supply chain. This supply chain is part of the actions required for information gathering and transmission to stakeholders. These activities enable those on the front lines to find the appropriate data, aid in creating policies and procedures to support data processing correctly and reliably and act as a framework to guarantee that the entire data supply chain is utilized to support the achievement of business objectives.
You May Like: Dollar Rental Car Government Rate
Data Governance: A Business Strategy
If data management is the logistics of data, data governance is the strategy of data. Data governance should feel bigger and more holistic than data management because it is: as an important business program, governance requires policy, best reached by consensus across the company.
The purpose of data governance is to provide tangible answers to how a company can determine and prioritize the financial benefits of data while mitigating the business risks of poor data. Data governance requires determining what data can be used in what scenarios which requires determining exactly what acceptable data is:
- What constitutes data?
- Where is it collected and used?
- How accurate must it be?
- Which rules must data follow?
- Who is involved in the various stages in a data lifecycle?
Importantly, data governance must go beyond IT and include stakeholders from across the enterprise. In order to ensure the safety, reliability, and trustworthiness of all data, governance requires that stakeholders from all business areas be involved. Consider the alternative: if each business silo approaches their data strategy differently, the end result is chaotic and not comprehensive enough to be useful.
The ultimate goal is to determine a holistic way to control data assets, so that the company can get the absolute most value from the data.
