Introduction
Welcome to the Simplitics PDQ documentation site. Each section offers comprehensive guides on managing and processing various components of the PDQ Platform. Each guide is designed to help you understand and implement best practices for different aspects of your data workflows.
PDQ by Simplitics
PDQ by Simplitics is a unified architecture with a pre-configured solution for automation in data management. The platform leverages cutting-edge tools to streamline data operations, ensuring efficiency and accuracy across the board.
PDQ integrates advanced data extraction, automated services, and robust quality frameworks, all designed to work seamlessly together.
Key Components
- AME (Active Metadata Engine): The central metadata repository and API hub. It stores all logic, records every operation that has run and holds every planned operation. Every other component reads its configuration from here.
- INGEST (Data Importer): An orchestrated data extraction tool for SQL, API, Message Queue and File-based data. It operates on a schedule and can detect and capture changes.
- DLS (Data Lake Service): An automated service that profiles, catalogs, archives, and publishes data in a standardized format.
- DWA (Data Warehouse Automation): A tool that incorporates best practices in data management and orchestration to automate data warehousing tasks.
- QPI (Quality Performance Indicator): A data quality framework that identifies and highlights data deviations in order to achieve overall better data quality for improved decision-making.
- DataOps Console: A console for managing data operations, simplifying daily tasks, and providing insights into platform performance.
- Developer UI: A user interface for adding custom logic or configurations, facilitating the needed configuration of all of the above components.
Each component is described layer by layer in the Platform guide, and every acronym used across these docs is collected in the Glossary.
Find your way around
These docs are organised around what you are trying to do, not around how the product is built.
| I want to… | Go to |
|---|---|
| Understand the ideas and walk through a first pipeline | Get started |
| Know how the architecture fits together, layer by layer | How PDQ works |
| Check whether a capability exists | Data management capabilities |
| Configure sources, contracts, models and mappings | Develop |
| Install and run the platform in my own environment | Deploy |
| Monitor the platform and fix what broke overnight | Daily operations |
| Look up an answer, a term or a known limitation | Knowledge base |
| See what changed in a release | Release notes |
New here? Read Fundamentals, then follow Your first pipeline end to end.