Enterprise Data, Governed on Azure Databricks
The validated reference architecture for the QuickLaunch Foundation Pack on Azure Databricks. A complete lakehouse with automated pipelines, custom management tools, and pre-built Power BI semantic models, deployed in 8 to 12 weeks.
Reference Architecture
Data flows left to right. Source systems (ERP, CRM, HR, financial, and any SQL database, on premises or in the cloud, plus IoT devices and APIs) are ingested by QuickLaunch Replication, Databricks Lakeflow Connect, and Fivetran. Data lands in Azure Data Lake Storage Gen2 in Delta Lake format, then the medallion method refines it through Bronze, Silver, and Gold layers inside the Databricks workspace, processed with Serverless SQL and notebooks. Unity Catalog governs every layer. The Gold layer feeds pre-built Power BI semantic models, consumed in Power BI, Excel, and Databricks Genie. The same governed data supports AI and machine learning.
Architecture Components
| Component | Description |
|---|---|
| Data Sources | ERP (JD Edwards EnterpriseOne, NetSuite, Vista, SAP, Microsoft Dynamics, Oracle E-Business Suite), CRM (Salesforce, Microsoft Dynamics), HR (Workday, ADP), financial and corporate performance management (OneStream, Oracle Hyperion), and any SQL database (SQL Server, Oracle, PostgreSQL, MySQL, DB2). Cloud and on-premises systems, IoT device streams, and REST APIs are supported. |
| Data Movement | QuickLaunch Replication (automated change data capture, full and incremental syncs for SQL databases), Databricks Lakeflow Connect (native connector ingestion), and Fivetran (pre-built connectors for SaaS applications and cloud platforms). |
| Data Storage | Azure Data Lake Storage (ADLS) Gen2 with hierarchical namespace, storing data in Delta Lake format (ACID transactions, time travel, schema evolution). Organized with the medallion method: Bronze (raw), Silver (cleansed), Gold (business ready). |
| Data Processing | The Azure Databricks workspace runs the transformations. Databricks SQL Serverless provides instant-on, automatically scaled SQL compute. Notebooks (Python, SQL, Scala, R) support data engineering and data science. |
| QLA Management Tools | Delta Editor (visual lakehouse administration and ETL metadata generation), Schema Compare (environment comparison and controlled promotion across development, test, and production), the QuickLaunch Wheel File (reusable Python utilities for audit, logging, and Power BI integration), Lakehouse Audit (automated row-count, aggregate, and duplicate validation), and Lakehouse Security (row-level security and Microsoft Entra ID group mapping). |
| Data Modeling | Pre-built Power BI semantic models with star schemas, defined measures, and row-level security. Domain models for Finance, Supply Chain, Construction, and Human Resources. |
| Consume | Power BI (interactive dashboards, paginated reports, mobile, natural language Q&A), Excel (direct connectivity for pivot analysis), and Databricks Genie (natural language querying over governed data). |
Integration and Connectivity
- JDBC and ODBC through the Simba driver, with OAuth authentication.
- Databricks Lakeflow Connect for native ingestion.
- Fivetran for pre-built SaaS and cloud connectors.
- Power BI connector for semantic-model connectivity to the lakehouse, in Import and Direct Lake modes.
Unity Catalog is the governance layer across all data and AI assets.
Cloud Support
QuickLaunch is delivered on Azure Databricks today, and the reference architecture on this page reflects an Azure deployment. All QuickLaunch Databricks code and logic runs on every Databricks cloud, so AWS and GCP deployments are also possible, with more of the networking configuration handled on the customer side. To scope an AWS or GCP deployment, talk to your QuickLaunch sales representative.
Security and Governance
- Unity Catalog for centralized governance: row-level and column-level security, data lineage, and audit logging.
- Microsoft Entra ID for single sign-on and group-based access.
- Azure Key Vault for secrets management.
- Row-level security applied in the Power BI semantic model and on lakehouse materialized views, so access rules are enforced at both the model layer and the data layer.
Deployment
QuickLaunch follows a four-phase method. A typical deployment runs 8 to 12 weeks.
| Phase | Duration | Focus |
|---|---|---|
| 1. Discovery and Architecture | 1 to 2 weeks | Source-system review, security and catalog design, and target architecture. |
| 2. Foundation Implementation | 2 to 3 weeks | Stand up the Azure Databricks lakehouse, Unity Catalog, storage, and the QLA management tools. |
| 3. Data Engineering | 3 to 4 weeks | Build the ingestion pipelines and the Bronze, Silver, and Gold transformations. |
| 4. Analytics and Deployment | 2 to 3 weeks | Deploy the Power BI semantic models, apply row-level security, validate, and hand over. |
Leading Enterprises Succeeding with QuickLaunch and Databricks
Frequently Asked Questions
Which clouds does QuickLaunch support for Databricks?
QuickLaunch is delivered on Azure Databricks today, and the reference architecture reflects an Azure deployment into your own Azure tenant with ADLS Gen2 storage. All QuickLaunch Databricks code and logic runs on every Databricks cloud, so AWS and GCP deployments are also possible, with more of the networking configuration handled on the customer side. To scope an AWS or GCP deployment, talk to your QuickLaunch sales representative.
How does QuickLaunch connect to Azure Databricks?
Through JDBC and ODBC using the Simba driver with OAuth authentication, Databricks Lakeflow Connect, Fivetran, and the Power BI connector (Import and Direct Lake modes). Unity Catalog governs access across all of them.
How is data secured and governed in this architecture?
Unity Catalog provides row-level and column-level security, data lineage, and audit logging across all data and AI assets. Microsoft Entra ID handles single sign-on and group-based access, Azure Key Vault manages secrets, and row-level security is enforced in the Power BI semantic model and on lakehouse materialized views.
How long does a deployment take?
A typical QuickLaunch Foundation Pack deployment on Azure Databricks runs 8 to 12 weeks across four phases: Discovery and Architecture, Foundation Implementation, Data Engineering, and Analytics and Deployment.
How does licensing work? Do we need to buy Databricks separately?
Yes. Databricks and Power BI licenses are purchased separately from QuickLaunch. You will need an Azure Databricks workspace with compute and Power BI Premium or Fabric capacity. QuickLaunch provides the Foundation Pack (lakehouse infrastructure and management tools) and optional Application Packs for systems like JD Edwards, Vista, NetSuite, and OneStream. We help size the Databricks environment and estimate costs during the Discovery and Architecture phase.
What makes Delta Editor and Schema Compare different from native Databricks capabilities?
Delta Editor provides a visual interface for lakehouse administration that removes the need for manual SQL scripting, letting analysts generate ETL metadata, create Delta Lake tables, and manage the lakehouse without coding. Schema Compare synchronizes environments (development, test, production) and identifies schema differences, a capability not available in native Databricks that prevents deployment errors. Together they reduce manual lakehouse management effort and extend data engineering tasks beyond specialist teams.
Can we combine data from multiple applications like JD Edwards and Salesforce?
Yes. The Foundation Pack creates a unified data environment where Application Packs load data from different systems into the same lakehouse. You can combine JD Edwards financial data with Salesforce opportunities, link Vista project costs to NetSuite revenue, or integrate OneStream budgets with JD Edwards actuals in a single Power BI report. Common dimensions (customers, products, time) are standardized across systems, enabling cross-application analysis that is difficult with siloed data.
Ready to Ignite Your Enterprise Intelligence?
QuickLaunch acts as an extension of your team. From onboarding and training to long-term support, we’re with you every step of the way.
