Clearpill

Overview

Clearpill is Seraphnet's premier GenAI app designed to eliminate ideological bias and fact-check content. It is currently in the proof-of-concept (PoC) phase, representing Seraphnet's first foray into the world of GenAI applications. Clearpill is part of Seraphnet's network of GenAI apps engineered to provide unbiased and factual information.

Architecture

Clearpill's backend architecture is built upon a robust foundation, leveraging cutting-edge technologies and innovative approaches to ensure optimal performance and scalability.

Data Warehouse

The core of Clearpill's backend is a Data Warehouse powered by PostgreSQL, a renowned open-source relational database management system. This Data Warehouse serves as a reliable and efficient storage solution for the vast amounts of data that fuel the application's core functionalities. Seraphnet plans to migrate to pgVector, a vector database tailored for high-performance analytics and machine learning workloads, to further enhance the app's capability to analyze and process text data without bias.

Data Pipeline

Clearpill's data processing is orchestrated by Kedro, a powerful open-source framework for building reproducible data pipelines. Kedro provides a modular and extensible architecture, allowing developers to create and manage complex data pipelines with ease. The data pipeline is further enhanced by the integration of three powerful plugins:

1. Docker: Enables seamless deployment and management of Kedro pipelines within Docker containers, ensuring efficient scalability and portability across different environments.

2. Pandera: Simplifies the process of creating and managing data quality checks, ensuring the integrity and reliability of the data used for unbiased text analysis.

3. Airflow Astronomer: Provides a robust and scalable workflow management system, enabling efficient scheduling, monitoring, and orchestration of complex data pipelines.

To optimize performance, Clearpill leverages GPU acceleration by converting pandas DataFrames to cuDF, a GPU DataFrame library that provides significant speedups for data processing tasks.

API Layer

Clearpill's backend is equipped with FastAPI, a modern, high-performance web framework for building APIs with Python. FastAPI serves as the JSON API response layer, facilitating seamless communication between the backend and the frontend components. Its streaming capabilities enable efficient handling of responses from Large Language Models (LLMs) used for text analysis and bias removal, reducing perceived latency and providing a more responsive user experience.

Monitoring and Logging

Seraphnet employs a robust monitoring and logging system to maintain the health and performance of the Clearpill backend. This system collects and analyzes various metrics, such as CPU and memory usage, network traffic, and response times, providing real-time insights and enabling proactive identification and resolution of potential issues. The logging system captures detailed logs from various components of the backend, providing valuable diagnostic information for troubleshooting and analysis.