Data Engineering for Digital Health Apps

Omniwyse experts help overcome the unique data engineering challenges faced by digital health companies.

Unified Data Processing

Seamless integration of batch and streaming data on a single platform, enabling scalable, real-time analytics essential for patient care and operational efficiency.

Tool Integration Flexibility

Open, adaptable platforms that support the best tools for data ingestion, transformation, and orchestration, tailored for healthcare ecosystems.

Efficient Data Ingestion

Simplified and automatic data ingestion mechanisms that support the dynamic nature of healthcare data, from electronic health records to IoT device outputs.

Automated ETL

Declarative frameworks for building and managing ETL and ML pipelines, reducing the complexity of transforming raw healthcare data into actionable insights.

Workflow Orchestration & Monitoring

Managed orchestration and observability services enhance data, analytics, and AI workflow reliability for digital health apps.

Advanced Reporting & Data Processing

Get high-performance engines that support large-scale analyses and comprehensive reporting, essential for complex healthcare data.

Legacy System Modernization

Resolve scalability, security and integration issues causing bottlenecks in data pipelines.

Manual Process Automation

Address the inefficiencies and error-prone nature of manual data entry and validation in healthcare with expertly engineered automation.

Featured Work

Expert Solutions for the Unique Challenges of Health Data Engineering

Architecting a multi-source ETL pipeline

Problem

This project faced two major challenges:

The client needed to develop a fully automated Big Data pipeline for data ingestion, transformation and load. The architecture had to support all types of file formats, sources, and targets, transforming one file into many tables and many tables into one file.

Resolution

The legacy Perl and Shell script data ingestion processes were migrated to Python, where trillions of records are now being stored on a daily basis. The new process we implemented is scalable and automated, using both Python and Airflow. It supports all kinds of source file formats, database and multipart files, processing billions of records on a regular basis using a failsafe, scalable architecture.

Automating KPI generation and business reports

Problem

In response to user expectations and the demands of a rapidly changing data landscape, the client recognized the necessity for modern platforms to supersede their outdated FRED server. They required solutions capable of automating the recording and reporting of KPI values assigned to each customer at regular, predefined intervals.

Resolution

The initial step involved transitioning the KPI process from the old FRED Server to Airflow, utilizing Python, SQL, and Airflow-based scripts. This transition facilitated the automated generation of KPI values for reports. Subsequently, the automation of various business reports (QBR, ABR, etc.) was accomplished. To accommodate future development and testing needs as new requirements emerged, our engineering team was made readily available to provide ongoing support to the client.

Identifying and reporting care gaps important to a user's health

Problem

Healthcare consumers often lack awareness of potential omissions in their medical treatment plans. The challenge was to devise a method for pinpointing care gaps for each user, guided by clinical team-defined data rules. An automated system was necessary to apply these rules and determine the presence of care gaps, along with recommendations for consumers to address them.

Resolution

We built an automated solution that identifies care gaps for each user, comprising four key steps:

Uploading clinical rules, codes, and codesets from an AWS S3 bucket to Vertica database through a file load process.
Preprocessing the data.
Generating member tags to map a user against the applicable codeset
Creating member rules based on clinical guidelines. This system spots any care gaps for a user and sends appropriate notifications.

Automating report and document generation to track patient health data

Problem

Initially, the client's report generation process was entirely manual, posing significant challenges due to the extensive changes required for even minor adjustments. To address this inefficiency, a new architecture supporting a metric ID system was needed.

Resolution

With Python, the processing time has been drastically reduced. Where generating all QBRs previously took 24-26 hours, it now only requires 1.5 hours, eliminating manual intervention. This efficiency gain comes with the need for fewer resources on specific documents, while also offering high customizability. The automation process now generates nearly all documents, significantly enhancing efficiency.

Implementing the data pipeline for a new consumer benefits feature

Problem

Most consumers were unaware of the full structure of their employee benefit plans, including available benefits and the costs of procedures. Navigating health plans was often a challenge.

Resolution

We built a new Advocacy Application that now assigns an Advocate to each consumer to assist in managing their benefit plan more effectively. An end-to-end ETL pipeline has been developed to extract, process, and organize user benefit structures and advocacy data, storing it in the target Postgres database for direct use by the Advocacy Application.

Developing an internal dashboard for managing Data Ops

Problem

The initial version of the Airflow UI in use was deficient in features for segregating data by teams, and it also lacked the necessary permissions for diverse teams and roles within the company. This setup posed challenges, especially for those unfamiliar with operating Airflow at such a detailed level. The client requested the development of an Internal Dashboard to facilitate job tracking, monitoring, and triggering by their team.

Resolution

We developed a new Dashboard, featuring a user interface accessible to individuals without a development background in Airflow. This Dashboard is capable of displaying jobs from various platforms, including Airflow and Cron, categorizing them by teams, and incorporating all user actions associated with Airflow. It also presents job statistics, such as counts and error messages, to aid in troubleshooting as required.

Data Engineering for Digital
Health Apps

Featured Work

Expert Solutions for the Unique Challenges of Health Data Engineering

Architecting a multi-source ETL pipeline

Automating KPI generation and business reports

Identifying and reporting care gaps important to a user's health

Automating report and document generation to track patient health data

Implementing the data pipeline for a new consumer benefits feature

Developing an internal dashboard for managing Data Ops

The money spent on patching up old tech could be invested in innovation.

Technologies We Use

Omniwyse, Your Trusted Partner

Talk to a real person about your needs.

Data Engineering for DigitalHealth Apps

Featured Work

Expert Solutions for the Unique Challenges of Health Data Engineering

Architecting a multi-source ETL pipeline

Automating KPI generation and business reports

Identifying and reporting care gaps important to a user's health

Automating report and document generation to track patient health data

Implementing the data pipeline for a new consumer benefits feature

Developing an internal dashboard for managing Data Ops

The money spent on patching up old tech could be invested in innovation.

Technologies We Use

Omniwyse, Your Trusted Partner

Talk to a real person about your needs.

Data Engineering for Digital
Health Apps