Planning Your Enterprise AI Investments? Discover How to Choose the Right AI Use Case →
Search through our services, key insights, resources, stories, blogs and case studies.
Search through our services, key insights, resources, stories, blogs and case studies.
Search through our services, key insights, resources, stories, blogs and case studies.
Search through our services, key insights, resources, stories, blogs and case studies.
Reference workflows, reusable notebooks, and implementation guidance to apply configurable retention rules in Databricks, protect records on hold, and mark eligible records as archived with decision logs and run details.
Trusted By :










Retention rules need to account for both record age and conditions that require records to remain available. Maintaining these rules in separate scripts can make them harder to update, apply consistently, and review.
DataTheta’s Data Retention & Archival Framework reads settings from one JSON configuration file and applies KEEP, HOLD, or ARCHIVE decisions. It updates record statuses, captures archival details, and exposes active and archived records through separate views, with held records included in the active view.
AI systems in production
Avg. time to first outcome
Forecast accuracy improvement
Faster decision cycles
Revenue influenced by AI
Manual processing eliminated
Define retention settings, the date column for record age, and hold conditions in one JSON file. Update rules without changing the main pipeline code.
Evaluate record age using a fixed as_of_date or today’s date. Apply KEEP, HOLD, or ARCHIVE decisions according to the configured rules.
Protect records matching configured hold conditions from archival. Keep held records available in the active view alongside other retained records.
Mark eligible records ARCHIVED and capture the archive date, run ID, and applied rule. Expose archived records through a dedicated archive view.
Review active and archived datasets alongside pipeline record counts, processing mode, and execution status. Use the results notebook to inspect each run’s outputs.
Explore reusable accelerators for data retention, governance, data quality, reconciliation, synchronization, and Databricks data engineering.
Use DataTheta’s Data Retention & Archival Framework to apply retention rules, protect held records, and review archival decisions and pipeline results in Databricks.
DataTheta is an enterprise Data, Analytics, and AI consulting company that helps organizations build AI-ready data foundations through Data Engineering, Data Science, Business Intelligence, Data Warehousing, Generative AI, and On-Demand Experts.
©2026 Copyright DataTheta – Lance Labs Inc.
Tell us about your business goals, data challenges, or technology requirements. Our team will review your enquiry and connect with you to discuss the right solution.
No spam. Your information stays private.