Costa Rica
Last updated: 2025-07-17
List of References (Click to expand)
- What's new in Microsoft Purview
- Pricing - Microsoft Purview
- Pricing guidelines for classic Microsoft Purview data governance
- Purview how to upgrade from free to enterprise
- What's available in the free version of Microsoft Purview governance solutions?
- Applications available in free vs enterprise
- Set up authentication using service principal
- How to get lineage from Airflow into Microsoft Purview (Preview)
- Microsoft Purview Data Catalog lineage user guide
Table of Content (Click to expand)
Purview Free: Provides basic data governance capabilities, suitable for small-scale or initial exploration of Purview’s features. It includes basic cataloging, limited data discovery, and basic compliance tools.
Purview Enterprise: Offers comprehensive data governance, protection, and compliance features. It supports a wide range of data sources, advanced classification, full DLP, information protection, compliance management, and seamless integration with Azure services.
| Feature | Purview Free | Purview Enterprise |
|---|---|---|
| Data Catalog | Basic cataloging capabilities. Limited to 1,000 annotated assets. |
Full cataloging capabilities. No limit on the number of annotated assets. |
| Data Discovery | Limited to Azure and Microsoft Fabric resources. Auto discovery of Azure data sources. |
Supports a wide range of data sources, including on-premises, multicloud, and SaaS applications. Automated scans for the hybrid data estate. |
| Data Lineage | Basic lineage tracking for a limited set of data sources. | Comprehensive lineage tracking across all supported data sources. |
| Data Classification | Basic classification capabilities. Definition and manual application of classifications and terms. |
Advanced classification with automatic labeling and sensitivity labels. Automated application of classifications and terms. |
| Data Loss Prevention (DLP) | Not included. | Full DLP capabilities to prevent unauthorized sharing of sensitive information. |
| Information Protection | Not included. | Includes encryption and access controls to protect sensitive data. |
| Compliance Management | Basic compliance tools. | Comprehensive compliance management, including Compliance Manager and audit capabilities. |
| Data Quality | Basic data profiling. | Advanced data quality features, including quality rules and continuous monitoring. |
| Insider Risk Management | Not included. | Full insider risk management capabilities to detect and respond to potential data leaks. |
| eDiscovery | Not included. | Full eDiscovery capabilities for legal and compliance investigations. |
| Integration with Azure Services | Limited integration with Azure services. | Seamless integration with a wide range of Azure services, including Synapse Analytics, SQL, and Power BI. |
| Data Map | Basic data map capabilities. Manual creation of assets using the data map APIs. |
Full data map with detailed visualizations and relationship tracking. Full use of Microsoft Purview's REST APIs. |
| Monitoring and Reporting | Basic monitoring and reporting. | Advanced monitoring and reporting, including Data Estate Insights. |
| User Access | Limited to data curators. Role group access control to platform and apps. |
Full access for all users, including data stewards and analysts. Fine-grained, collection-level access control to platform and apps. |
| Support and SLA | Community support. | Enterprise-grade support and SLA. |
| Workflows | Not included. | Included. |
| Business Rules | Not included. | Included. |
| Support for Business Assets and Managed Attributes | Not included. | Included. |
| Descriptions, Tags, and Contacts | Manual descriptions, tags, and contacts. | Manual and bulk descriptions, tags, and contacts. |
Keypoints of Microsoft Purview:
Integration with Microsoft Ecosystem: Purview offers deep integration with Azure, Power BI, and Microsoft 365, providing a seamless experience for organizations already using these tools.Advanced Governance and Compliance: Purview provides robust governance and compliance features, ensuring your data management practices meet regulatory standards.AI-Powered Search and Discovery: With AI-driven capabilities, Purview enhances data discovery and classification, making it easier to find and manage data assets.Enterprise-Grade Security: Leveraging Azure's security infrastructure, Purview offers top-notch security features to protect your data.Cost-Effective: For organizations using Azure, Purview can be a more cost-effective solution due to its deep integration with Azure services.
| Feature/Tool | Microsoft Purview | Atlan | Alation | Collibra | Informatica |
|---|---|---|---|---|---|
| Data Lineage | End-to-end lineage, integrates with Azure Data Factory, supports column-level and cross-system lineage | Visual lineage maps, supports column-level lineage, tracks data transformations and dependencies | Column-level lineage, impact analysis, automated lineage extraction | End-to-end lineage, automated data flow tracking, supports both technical and business lineage | Comprehensive lineage with data transformation tracking, supports column-level lineage |
| Integration | Deep integration with Azure services, Power BI, Microsoft 365, supports various data sources and ETL tools | Integrates with Snowflake, Databricks, AWS, GCP, BI tools, and more | Integrates with BI tools, databases, cloud services, and data lakes | Extensive integrations with enterprise systems, cloud platforms, and BI tools | Broad integration support including cloud and on-premises systems, supports ETL tools |
| Data Governance | Advanced governance, integrates with Azure Policy, Microsoft Information Protection, role-based access control, compliance tracking | Role-based access control, data quality rules, policy management | Policy management, data stewardship, role-based access control, data quality monitoring | Data stewardship, policy management, data quality rules, compliance tracking | Data quality management, policy enforcement, role-based access control, compliance tracking |
| Data Discovery | AI-powered search and discovery, integrates with Azure Cognitive Services, metadata tagging, data profiling | AI-powered search with NLP, metadata tagging, data profiling | Advanced search with ML, metadata tagging, data profiling | Comprehensive search with metadata tagging, data profiling, and classification | AI-driven data discovery, metadata management, data profiling |
| Collaboration | Integrated collaboration tools, supports Microsoft Teams, data annotations, and sharing | Collaborative workspace, supports team-based projects, data annotations | Collaborative features, data stewardship workflows, data annotations | Collaboration tools, data sharing, annotations, and workflows | Collaborative environment, data sharing, version control, data annotations |
| Security | Enterprise-grade security, integrates with Azure Security Center, encryption, access controls, compliance certifications | Encryption, access controls, audit logs, compliance certifications | Strong security features, audit trails, compliance certifications | Robust security, data masking, encryption, compliance certifications | High security standards, data masking, encryption, audit logs |
| Scalability | Highly scalable with Azure, supports large-scale data environments, elastic scaling | Cloud-native architecture, supports large-scale deployments | Scalable architecture, supports large datasets | Distributed architecture, supports large-scale deployments | Cloud and on-premises support, highly scalable |
| Cost | Cost-effective for Azure users, pay-as-you-go model, subscription-based options | Competitive pricing, flexible plans, usage-based pricing | Premium pricing, enterprise features, subscription-based | Premium pricing, extensive features, subscription-based | Premium pricing, comprehensive support, subscription-based |
Microsoft Purview offers a flexible pricing model based on several components:
| Pricing Aspect | Description | Details |
|---|---|---|
| Data Map Population | Charges based on the number of data assets scanned and classified | - Cost Basis: Number of data assets scanned - Usage: Suitable for organizations with varying data volumes - Example: Scanning 10,000 data assets may incur different costs compared to 50,000 data assets |
| Data Map Enrichment | Costs associated with processing information to optimize storage and generate insights | - Cost Basis: Processing and enrichment activities - Usage: Enhances data quality and insights - Example: Enriching metadata for better searchability and classification |
| Subscription-Based Licensing | Some capabilities offered as subscription-based, with costs varying based on usage and specific features | - Cost Basis: Subscription fees based on selected features - Usage: Predictable costs for budgeting - Example: Subscribing to advanced governance features or AI-powered search capabilities |
| Pay-As-You-Go | Pricing is scalable and depends on the number of data assets scanned | - Cost Basis: Actual usage - Usage: Flexible and scalable, suitable for businesses of all sizes - Example: Paying for the exact number of data assets scanned each month |
Scalability: Purview's pay-as-you-go model allows for cost-effective scaling based on actual usage, which can be more economical for organizations with fluctuating data needs.Integration with Azure: For organizations already using Azure, Purview can offer additional cost savings due to its deep integration with Azure services.
| Aspect | Microsoft Purview | Atlan | Alation | Collibra | Informatica |
|---|---|---|---|---|---|
| Pricing Structure | Pay-As-You-Go, Subscription-Based | Tiered Pricing (Starter, Premier, Enterprise) | Subscription-Based, Custom Pricing | Subscription-Based, Annual Licensing | Consumption-Based, IPU (Informatica Processing Unit) Pricing |
| Cost | Scalable based on usage | Starts at $500/month for Starter Plan | Starts at $60,000/year. Click here for more information | Starts at $170,000/year. Click here for more information | Flexible, based on usage. Click here for more information |
| Customization | Flexible, based on data assets scanned | Custom pricing for Enterprise Plan | Custom pricing based on features and usage | Custom pricing based on contract term | Customizable based on IPU consumption. Click here for more information |
| Trial Options | Free trial available | Free trial for Starter and Premier Plans. Click here for more information | Free trial available upon request | Free demo available. Click here for more information | Free trial available. Click here for more information |
Find below different scenarios to manage data governance, protection, and compliance:
| Scenario | Description | Steps |
|---|---|---|
| Data Governance for a Financial Institution | Demonstrates how Purview can be used to discover, catalog, and manage data assets, ensuring data quality and compliance in a financial setting. | 1. Data Discovery: Scan all data sources (e.g., SQL databases, Azure Blob Storage) to discover and catalog data assets. 2. Metadata Collection: Collect metadata and classify data based on sensitivity (e.g., PII, financial data). 3. Data Cataloging: Register discovered data assets in the Purview Data Catalog for easy search and access. 4. Data Lineage: Capture and visualize data lineage to understand data flow and transformations. 5. Data Stewardship: Assign data stewards to manage and curate data assets, ensuring data quality and compliance. 6. Compliance Management: Implement governance policies and use compliance tools to meet regulatory requirements (e.g., GDPR, CCPA). 7. Monitoring and Reporting: Continuously monitor data quality and generate compliance reports. |
| Data Protection for a Healthcare Provider | Shows how Purview helps classify and protect sensitive healthcare data, implement data loss prevention policies, and ensure compliance with healthcare regulations. | 1. Data Classification: Automatically classify sensitive data (e.g., patient records) using Purview's classification tools. 2. Data Loss Prevention (DLP): Implement DLP policies to prevent unauthorized sharing of sensitive information. 3. Information Protection: Apply encryption and access controls to protect sensitive data throughout its lifecycle. 4. Insider Risk Management: Monitor and manage insider risks by detecting and responding to potential data leaks or policy violations. 5. Audit and Compliance: Use audit tools to track user activities and changes, ensuring compliance with healthcare regulations (e.g., HIPAA). |
| Data Analytics for an E-commerce Company | Highlights how Purview supports data ingestion, processing, and analytics, while maintaining data lineage, quality, and governance in an e-commerce environment. | 1. Data Ingestion: Ingest data from various sources (e.g., web servers, sales databases) into Azure Synapse Analytics. 2. Data Processing: Use Azure Synapse to process and transform data for analytics. 3. Data Lineage: Capture data lineage to understand the flow and transformations of data from source to analytics. 4. Data Cataloging: Register processed data assets in the Purview Data Catalog for easy discovery and access. 5. Data Quality: Profile data to assess its quality and define quality rules to maintain high standards. 6. Reporting and Visualization: Use Power BI to create reports and dashboards, ensuring data governance and security. |
| Compliance Management for a Global Enterprise | Illustrates how Purview can be used to manage compliance across a global organization, including data discovery, classification, policy implementation, and continuous monitoring. | 1. Data Discovery: Scan and catalog data assets across on-premises and cloud environments. 2. Data Classification: Classify data based on sensitivity and regulatory requirements. 3. Compliance Assessments: Use Purview Compliance Manager to perform assessments and identify compliance gaps. 4. Policy Implementation: Implement data governance and compliance policies to address identified gaps. 5. Continuous Monitoring: Continuously monitor compliance status and generate reports for regulators and stakeholders. 6. Incident Response: Use audit and investigation tools to respond to compliance incidents and mitigate risks. |
- Set Up Microsoft Purview Account
- Sign Up:
- Go to the Microsoft Purview portal.
- Click on
Start freeorGet startedto sign up for an account.
- Create Purview Account:
- In the Azure portal, search for
Purview. - Click on
Createto set up a new Purview account. - Fill in the required details such as subscription, resource group, and account name.
- Choose the region and pricing tier.
- Review and create the account.
- In the Azure portal, search for
- Sign Up:
- Data Domain:
Note
Remember to add purview identity with required RBAC (e.g admin) within the data source desired:
Also add the required roles within the Purview portal, for each collection>
-
Data Discovery
-
Navigate to Purview Portal: Access the Purview portal from the Azure portal by selecting your Purview account.
-
Register Data Sources:
-
Go to the
Data Mapsection. -
Click on
Registerand select the data source type (e.g., Azure SQL Database, Azure Blob Storage). -
Provide connection details:
- For Azure SQL Database: Enter the server name, database name, and authentication details.
- For Azure Blob Storage: Enter the storage account name and access key.
-
-
Configure Scan Settings:
- Set the scan scope: Select specific databases, tables, or containers to scan.
- Schedule the scan frequency: Choose how often the scan should run (e.g., daily, weekly).
- Configure advanced settings: Set up filters to include or exclude specific data assets.
-
Initiate Scan:
- Click on
Start Scanto begin the discovery process. - Monitor the scan progress and review the results once completed.
- Click on
-
-
Metadata Collection
-
Review Discovered Assets: Go to the
Data Mapsection and review the list of discovered data assets.
-
Automatic Classification:
- Go to the
Classificationssection. - Enable automatic classification:
- Select the classifiers you want to apply (e.g., PII, financial data).
- Configure classification rules if needed.
- Go to the
-
Manual Metadata Addition:
- Edit metadata for specific data assets:
- Click on a data asset to view its details.
- Add custom tags, descriptions, and contact information for data stewards.
- Edit metadata for specific data assets:
-
-
Data Cataloging
- Register Data Assets: Ensure all scanned assets are listed in the
Data Catalogsection. - Organize Data Assets:
- Add relevant data assets to the collection.
- Define glossary terms:
- Go to the
Glossarysection. - Click on
New Termto create glossary terms. - Link glossary terms to data assets.
- Go to the
- Add Descriptions, Tags, and Contacts:
- Edit each asset:
- Click on a data asset to view its details.
- Add detailed descriptions, relevant tags, and contact information for data stewards.
- Edit each asset:
- Register Data Assets: Ensure all scanned assets are listed in the
-
Data Lineage
- Enable Lineage Tracking:
- Go to the
Lineagesection. - Ensure lineage tracking is enabled for the registered data sources.
- Go to the
- Visualize Data Flow:
- Use lineage visualization tools:
- Navigate to the
Lineagesection. - Select a data asset to view its lineage.
- Explore the data flow and transformations.
- Navigate to the
- Use lineage visualization tools:
- Analyze Lineage:
- Identify upstream and downstream dependencies:
- Use the lineage graph to trace data dependencies.
- Analyze the impact of changes to data assets.
- Identify upstream and downstream dependencies:
- Enable Lineage Tracking:
-
Data Stewardship
- Assign Data Stewards:
- Go to the
Rolessection. - Assign data steward roles:
- Select a collection or data asset.
- Assign users as data stewards.
- Go to the
- Monitor Data Quality and Governance:
- Set up dashboards:
- Go to the
Monitoringsection. - Create dashboards to track data quality metrics and governance activities.
- Go to the
- Set up dashboards:
- Curate Data Assets:
- Regularly review and update metadata, classifications, and lineage information:
- Schedule periodic reviews.
- Make necessary updates to ensure data accuracy and compliance.
- Regularly review and update metadata, classifications, and lineage information:
- Assign Data Stewards:
-
Compliance Management
- Implement Governance Policies:
- Go to the
Policiessection. - Define and apply data governance policies:
- Create new policies for data retention, access controls, and usage.
- Apply policies to relevant data sources.
- Go to the
- Configure Data Retention and Access Policies:
- Set up policies to manage data lifecycle and access permissions:
- Define retention periods for different data types.
- Configure access controls to restrict data access based on roles.
- Set up policies to manage data lifecycle and access permissions:
- Use Compliance Manager:
- Perform compliance assessments:
- Go to the
Compliance Managersection. - Select relevant compliance templates (e.g., GDPR, CCPA).
- Run assessments and review the results.
- Go to the
- Generate compliance reports: Create reports to document compliance status and actions taken.
- Perform compliance assessments:
- Implement Governance Policies:
-
Monitoring and Reporting
- Set Up Monitoring Dashboards:
- Go to the
Monitoringsection. - Create dashboards:
- Select metrics to monitor (e.g., data quality, compliance status).
- Configure visualizations and alerts.
- Go to the
- Continuous Monitoring:
- Use real-time alerts and notifications:
- Set up alerts for data quality issues and compliance risks.
- Configure notifications to inform relevant stakeholders.
- Use real-time alerts and notifications:
- Generate and Review Compliance Reports:
- Regularly generate reports:
- Go to the
Reportssection. - Create and schedule reports to review compliance status and governance effectiveness.
- Go to the
- Regularly generate reports:
- Set Up Monitoring Dashboards:
- Data Classification
- Navigate to the Microsoft Purview portal:
- Access the Purview portal from the Azure portal by selecting your Purview account.
- Set up a new data source connection:
- Go to the
Data Mapsection. - Click on
Registerand select the data source type (e.g., Azure SQL Database, Azure Blob Storage). - Provide connection details:
- For Azure SQL Database: Enter the server name, database name, and authentication details.
- For Azure Blob Storage: Enter the storage account name and access key.
- Go to the
- Configure the scan settings and initiate the scan:
- Set the scan scope: Select specific databases, tables, or containers to scan.
- Schedule the scan frequency: Choose how often the scan should run (e.g., daily, weekly).
- Configure advanced settings: Set up filters to include or exclude specific data assets.
- Click on
Start Scanto begin the discovery process. - Monitor the scan progress and review the results once completed.
- Automatically classify sensitive data:
- Go to the
Classificationssection. - Enable automatic classification:
- Select the classifiers you want to apply (e.g., PII, patient records).
- Configure classification rules if needed.
- Go to the
- Navigate to the Microsoft Purview portal:
- Data Loss Prevention (DLP)
- Navigate to the DLP section in the Purview portal:
- Access the DLP section from the main menu.
- Create and configure DLP policies:
- Click on
Create Policy. - Define the policy name and description.
- Specify the conditions for the policy (e.g., detecting sensitive information in emails or documents).
- Set the actions to be taken when a policy violation is detected (e.g., block sharing, notify admin).
- Click on
- Apply DLP policies to relevant data sources:
- Select the data sources to which the DLP policy should apply.
- Save and activate the policy.
- Monitor for policy violations:
- Go to the
DLP Reportssection. - Review the reports to monitor for any policy violations and take necessary actions.
- Go to the
- Navigate to the DLP section in the Purview portal:
- Information Protection
- Navigate to the Information Protection section in the Purview portal:
- Access the Information Protection section from the main menu.
- Apply encryption and access controls:
- Click on
Create Label. - Define the label name and description.
- Configure encryption settings (e.g., specify who can access the data and what actions they can perform).
- Apply the label to sensitive data.
- Click on
- Monitor and manage access to sensitive data:
- Go to the
Label Activity Explorersection. - Review the activity logs to monitor access to sensitive data and ensure compliance with policies.
- Go to the
- Navigate to the Information Protection section in the Purview portal:
- Insider Risk Management
- Navigate to the Insider Risk Management section in the Purview portal:
- Access the Insider Risk Management section from the main menu.
- Configure policies to detect and respond to potential data leaks or policy violations:
- Click on
Create Policy. - Define the policy name and description.
- Specify the conditions for the policy (e.g., detecting unusual data access patterns).
- Set the actions to be taken when a policy violation is detected (e.g., alert admin, restrict access).
- Click on
- Monitor insider activities and investigate potential risks:
- Go to the
Risk Activity Explorersection. - Review the activity logs to monitor insider activities and investigate any potential risks.
- Go to the
- Navigate to the Insider Risk Management section in the Purview portal:
- Audit and Compliance
- Navigate to the Audit section in the Purview portal:
- Access the Audit section from the main menu.
- Configure audit settings to track user activities and changes:
- Click on
Create Audit Log. - Define the log name and description.
- Specify the activities to be tracked (e.g., data access, modifications).
- Set the retention period for the audit logs.
- Save and activate the audit log.
- Click on
- Generate audit reports to ensure compliance with healthcare regulations (e.g., HIPAA):
- Go to the
Audit Reportssection. - Create and schedule audit reports to review user activities and ensure compliance with regulations.
- Go to the
- Navigate to the Audit section in the Purview portal:
- Data Ingestion
- Set up data ingestion pipelines:
- Use Azure Data Factory or Azure Synapse Analytics.
- Create a new pipeline:
- In Azure Data Factory, go to the
Authorsection and click onNew pipeline. - In Azure Synapse Analytics, go to the
Integratesection and click onNew pipeline.
- In Azure Data Factory, go to the
- Add data source connectors:
- For web servers: Use the HTTP connector to ingest web server logs.
- For sales databases: Use the Azure SQL Database connector to ingest sales data.
- Ingest data from various sources:
- Configure the data source settings:
- For HTTP connector: Enter the URL and authentication details.
- For Azure SQL Database connector: Enter the server name, database name, and authentication details.
- Set up data sinks:
- Choose Azure Synapse Analytics as the destination.
- Configure the destination settings (e.g., database, table).
- Configure the data source settings:
- Monitor the data ingestion process:
- Go to the
Monitorsection in Azure Data Factory or Azure Synapse Analytics. - Check the pipeline runs to ensure data is being ingested correctly.
- Go to the
- Set up data ingestion pipelines:
- Data Processing
- Use Azure Synapse Analytics to process and transform the ingested data:
- Go to the
Datasection in Azure Synapse Analytics. - Create a new SQL script or Spark notebook to process the data.
- Go to the
- Apply necessary transformations and aggregations:
- Write SQL queries or Spark code to clean, transform, and aggregate the data.
- Example transformations:
- Remove duplicates.
- Aggregate sales data by product category.
- Enrich web server logs with geolocation data.
- Store the processed data:
- Save the processed data in a data warehouse or data lake:
- Use Azure Synapse Analytics to create tables in a dedicated SQL pool.
- Use Azure Data Lake Storage to store the processed data in Parquet or CSV format.
- Save the processed data in a data warehouse or data lake:
- Use Azure Synapse Analytics to process and transform the ingested data:
- Data Lineage
- Enable lineage tracking in Azure Synapse Analytics:
- Go to the
Managesection in Azure Synapse Analytics. - Enable data lineage tracking for the workspace.
- Go to the
- Capture data lineage:
- Ensure that all data processing activities are logged.
- Use built-in lineage tracking features to capture the flow and transformations of data.
- Visualize the data lineage in the Purview portal:
- Go to the
Lineagesection in the Purview portal. - Select a data asset to view its lineage.
- Explore the data flow and transformations.
- Go to the
- Enable lineage tracking in Azure Synapse Analytics:
- Data Cataloging
- Register processed data assets in the Purview Data Catalog:
- Go to the
Data Mapsection in the Purview portal. - Ensure that all processed data assets are listed.
- Go to the
- Organize data assets:
- Create collections:
- Go to the
Collectionssection. - Click on
New Collectionand provide a name and description. - Add relevant data assets to the collection.
- Go to the
- Define glossary terms:
- Go to the
Glossarysection. - Click on
New Termto create glossary terms. - Link glossary terms to data assets.
- Go to the
- Create collections:
- Add descriptions, tags, and contacts:
- Edit each asset:
- Click on a data asset to view its details.
- Add detailed descriptions, relevant tags, and contact information for data stewards.
- Edit each asset:
- Register processed data assets in the Purview Data Catalog:
- Data Quality
- Profile data to assess its quality:
- Go to the
Data Qualitysection in the Purview portal. - Select a data asset to profile.
- Review the data quality metrics (e.g., completeness, accuracy).
- Go to the
- Define and enforce data quality rules:
- Create data quality rules:
- Go to the
Data Quality Rulessection. - Click on
New Ruleand define the rule conditions (e.g., no null values, valid email format).
- Go to the
- Apply the rules to relevant data assets.
- Create data quality rules:
- Continuously monitor data quality:
- Set up data quality dashboards:
- Go to the
Monitoringsection. - Create dashboards to track data quality metrics.
- Go to the
- Address any data quality issues:
- Review the data quality reports and take corrective actions as needed.
- Set up data quality dashboards:
- Profile data to assess its quality:
- Reporting and Visualization
- Use Power BI to create reports and dashboards:
- Connect Power BI to Azure Synapse Analytics or Azure Data Lake Storage.
- Create new reports and dashboards:
- Use Power BI Desktop to design reports.
- Publish the reports to the Power BI service.
- Ensure data governance and security:
- Integrate Power BI with Purview:
- Go to the
Power BIsection in the Purview portal. - Enable data governance features for Power BI datasets.
- Go to the
- Integrate Power BI with Purview:
- Share reports and dashboards with stakeholders:
- Configure access controls:
- Set up roles and permissions in Power BI to control access to reports and dashboards.
- Share the reports and dashboards:
- Use the Power BI service to share reports with stakeholders while maintaining data security.
- Configure access controls:
- Use Power BI to create reports and dashboards:
- Data Discovery
- Scan and catalog data assets across on-premises and cloud environments using Purview:
- Navigate to the Microsoft Purview portal.
- Go to the
Data Mapsection.
- Set up data source connections and configure scan settings:
- Click on
Registerand select the data source type (e.g., Azure SQL Database, Azure Blob Storage, on-premises SQL Server). - Provide connection details:
- For Azure SQL Database: Enter the server name, database name, and authentication details.
- For Azure Blob Storage: Enter the storage account name and access key.
- For on-premises SQL Server: Use the self-hosted integration runtime to connect.
- Configure scan settings:
- Set the scan scope: Select specific databases, tables, or containers to scan.
- Schedule the scan frequency: Choose how often the scan should run (e.g., daily, weekly).
- Configure advanced settings: Set up filters to include or exclude specific data assets.
- Click on
- Initiate scans to discover and catalog data assets:
- Click on
Start Scanto begin the discovery process. - Monitor the scan progress and review the results once completed.
- Click on
- Scan and catalog data assets across on-premises and cloud environments using Purview:
- Data Classification
- Classify data based on sensitivity and regulatory requirements using Purview's classification tools:
- Go to the
Classificationssection. - Enable automatic classification:
- Select the classifiers you want to apply (e.g., PII, financial data, health data).
- Configure classification rules if needed.
- Go to the
- Apply sensitivity labels and classifications to discovered data assets:
- Review the automatically classified data.
- Manually apply or adjust sensitivity labels and classifications as needed.
- Review and refine classifications as needed:
- Regularly review the classifications to ensure accuracy.
- Update classification rules and labels based on new regulatory requirements or business needs.
- Classify data based on sensitivity and regulatory requirements using Purview's classification tools:
- Compliance Assessments
- Use Purview Compliance Manager to perform compliance assessments:
- Navigate to the
Compliance Managersection in the Purview portal. - Select relevant compliance templates (e.g., GDPR, CCPA, HIPAA).
- Configure the assessment settings and scope.
- Navigate to the
- Identify compliance gaps and areas for improvement:
- Run the compliance assessments.
- Review the assessment results to identify compliance gaps and areas for improvement.
- Generate compliance reports and review assessment results:
- Go to the
Reportssection. - Create and schedule compliance reports to document assessment results and actions taken.
- Go to the
- Use Purview Compliance Manager to perform compliance assessments:
- Policy Implementation
- Implement data governance and compliance policies using Purview's policy management tools:
- Go to the
Policiessection. - Click on
Create Policyand define the policy name and description. - Specify the policy conditions (e.g., data retention, access controls, usage).
- Go to the
- Configure data retention, access, and usage policies:
- Define retention periods for different data types.
- Configure access controls to restrict data access based on roles.
- Set usage policies to ensure data is used in compliance with regulations.
- Apply policies to relevant data sources and monitor for compliance:
- Select the data sources to which the policies should apply.
- Save and activate the policies.
- Monitor the policies to ensure compliance.
- Implement data governance and compliance policies using Purview's policy management tools:
- Continuous Monitoring
- Set up monitoring dashboards in the Purview portal:
- Go to the
Monitoringsection. - Create dashboards to track compliance status and data quality metrics.
- Go to the
- Continuously monitor compliance status and data quality:
- Use real-time alerts and notifications:
- Set up alerts for compliance issues and data quality risks.
- Configure notifications to inform relevant stakeholders.
- Use real-time alerts and notifications:
- Generate and review compliance reports regularly:
- Go to the
Reportssection. - Create and schedule reports to review compliance status and governance effectiveness.
- Go to the
- Set up monitoring dashboards in the Purview portal:
- Incident Response
- Use Purview's audit and investigation tools to respond to compliance incidents:
- Navigate to the
Auditsection in the Purview portal. - Configure audit settings to track user activities and changes.
- Use the
Investigationtools to analyze audit logs and identify incidents.
- Navigate to the
- Investigate potential compliance violations and take corrective actions:
- Review the audit logs to investigate potential compliance violations.
- Take corrective actions to address the violations.
- Document incident responses and review for continuous improvement:
- Document the incident responses and actions taken.
- Review the incident responses to identify areas for improvement and update policies as needed.
- Use Purview's audit and investigation tools to respond to compliance incidents:
This capability is currently in public preview and is achieved through integration with OpenLineage, an open framework for data lineage collection and analysis.
How it works:
- Enable OpenLineage in Airflow: By enabling OpenLineage in your Airflow instance, metadata and lineage information about jobs and datasets are automatically tracked as Directed Acyclic Graphs (DAGs) execute.
- Azure Event Hubs: The tracked metadata and lineage information are sent to an Azure Event Hubs instance that you configure.
- Microsoft Purview: Purview subscribes to the events from Azure Event Hubs, parses them, and ingests the metadata and lineage into the data map.
This integration supports capturing metadata such as:
- Airflow workspace
- Airflow DAG
- Airflow task

