pgsqlite - Load SQLite databases into PostgresSQL
Written by Nikos Vaggalis   
Monday, 17 October 2022

pgsqlite is a pure python module and command line tool that makes it simple to import a SQLite database into Postgres, saving a ton of time and hassle in the process.

sqlite

SQLite is amazing. In "In Praise Of SQLite" I describe why that is so. One of those reasons was extensibility:

in a sense extensibility is SQLite's biggest power; it adds unforeseen utility and value for its users, and we can witness many imaginative uses for it, provided by the community.

I then gave a list of such extensions.

But there's the other way too; not extending SQLite itself but extending third party products by means of SQLite. One such plugin is pgsqlite by Adam Fletcher of bit.io which allows for importing SQLite databases into Postgres seamlessly.

On paper such a migration might seem like a simple task, after all it's just structured data out and structured data in. It wasn't so in reality. Adam Fletcher found many obstacles in the way :

 

  • Typing Differences
    First, SQLite and Postgres use different types.
  • Identifier Quoting
    Mismatches in some cases between how PostgresSQL and Sqlite quote their objects.
  • Forgiving SQL Parsing
    SQLite is more lax with syntax errors in some SQL statements, while Postgres is not.

 

As such he had to make some tradeoffs like deciding what to do about NULLs on a per-type basis, recognizing that you may then need to modify the actual data in the database as you move it from SQLite to Postgres.

In the end, the tool was ready, simply invokable as :

pgsqlite. py [-h] -f SQLITE_FILENAME -p POSTGRES_CONNECT_URL [-d DEBUG] [--drop_tables DROP_TABLES] [--drop_everything DROP_EVERYTHING] [--drop_tables_after_import DROP_TABLES_AFTER_IMPORT]

Example : Import into the bit.io database adam/AMEND, with DEBUG-level logging.

python pgsqlite. py -f . . /example_dbs/Chinook_Sqlite.sqlite -p postgresql://adam:<password>@db.bit.io/adam/AMEND --debug true

That's it! Your data is owned by PostgresSQL now.

 

More Information

pgsqlite: A pure Python module to import SQLite databases into Postgres

Related Articles

Use Rust To Reduce The Size Of Your SQLite Database

In Praise Of SQLite

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


MongoDB Atlas Stream Processing Generally Available
13/05/2024

The MongoDB developers have announced that MongoDB Atlas now has support for stream processing. The news was announced at MongoDB.Local NYC.



OpenAI Introduces GPT-4o, Loses Sutskever
15/05/2024

It's an eventful week for OpenAI, the research company dedicated to making advances towards Artificial General Intelligence that are both safe and beneficial to all. A day after it showcased its lates [ ... ]


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Monday, 17 October 2022 )