lourousa/frauddetection

This project is an experimental demonstration of a "streamhouse" architecture, a system that bridges real-time data streaming with long-term data storage, built specifically to show how Apache Fluss (an incubating Apache project) can work alongside Apache Flink and Apache Iceberg for real-time fraud detection on bank transactions. The system processes transactions as they arrive, identifies suspicious ones using a Flink-based job, and then enriches those fraud records with account details by doing a live lookup against an account table. This enrichment step uses a temporal streaming lookup join, treating the account table as a reference dimension rather than a second stream. Once enriched, the fraud records flow into a long-term storage layer backed by an Iceberg table stored on MinIO, which is an S3-compatible object storage system. Metadata for the Iceberg tables is managed through an Iceberg REST catalog. Data is available at two speeds: hot (sub-second latency for recent transactions still in Fluss) and cold (minutes latency for historical records in the Iceberg layer). Three tables are created in Fluss: a transaction log, an account table that supports updates and deletes, and an enriched fraud log with data-lake tiering enabled. When tiering is enabled, Fluss automatically creates a matching Iceberg table and compacts data into it via a built-in Flink service, so historical records accumulate in the lakehouse without manual pipeline work. The project is deployed using Docker Compose and is written in Java. The README notes it is intended for exploration only and is not production-ready: cluster sizing and security configuration are minimal.