Athena SQLite is a project that allows you to query SQLite databases in S3 using Athena’s Federated Query functionality.
Install it from the Serverless Application Repository: AthenaSQLiteConnector.
Wait, what?!
SQLite in S3? Yea! As quite possibly the most prevelant database in the world, it’s not unsual for me to have various SQLite files laying around.
This Athena data connector allows you to query those databases directly from Athena.
Cool!
Right?! One of the fun things about this project is that SQLite is not intended to be a network database.
So I implemented a custom Virtual File System for S3 that could be used by the Python SQLite wrapper, APSW.