pgsqlite - Load SQLite databases into PostgresSQL
Written by Nikos Vaggalis   
Monday, 17 October 2022

pgsqlite is a pure python module and command line tool that makes it simple to import a SQLite database into Postgres, saving a ton of time and hassle in the process.

sqlite

SQLite is amazing. In "In Praise Of SQLite" I describe why that is so. One of those reasons was extensibility:

in a sense extensibility is SQLite's biggest power; it adds unforeseen utility and value for its users, and we can witness many imaginative uses for it, provided by the community.

I then gave a list of such extensions.

But there's the other way too; not extending SQLite itself but extending third party products by means of SQLite. One such plugin is pgsqlite by Adam Fletcher of bit.io which allows for importing SQLite databases into Postgres seamlessly.

On paper such a migration might seem like a simple task, after all it's just structured data out and structured data in. It wasn't so in reality. Adam Fletcher found many obstacles in the way :

 

  • Typing Differences
    First, SQLite and Postgres use different types.
  • Identifier Quoting
    Mismatches in some cases between how PostgresSQL and Sqlite quote their objects.
  • Forgiving SQL Parsing
    SQLite is more lax with syntax errors in some SQL statements, while Postgres is not.

 

As such he had to make some tradeoffs like deciding what to do about NULLs on a per-type basis, recognizing that you may then need to modify the actual data in the database as you move it from SQLite to Postgres.

In the end, the tool was ready, simply invokable as :

pgsqlite. py [-h] -f SQLITE_FILENAME -p POSTGRES_CONNECT_URL [-d DEBUG] [--drop_tables DROP_TABLES] [--drop_everything DROP_EVERYTHING] [--drop_tables_after_import DROP_TABLES_AFTER_IMPORT]

Example : Import into the bit.io database adam/AMEND, with DEBUG-level logging.

python pgsqlite. py -f . . /example_dbs/Chinook_Sqlite.sqlite -p postgresql://adam:<password>@db.bit.io/adam/AMEND --debug true

That's it! Your data is owned by PostgresSQL now.

 

More Information

pgsqlite: A pure Python module to import SQLite databases into Postgres

Related Articles

Use Rust To Reduce The Size Of Your SQLite Database

In Praise Of SQLite

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Meet Stretch - A Mobile Manipulator Robot
24/02/2024

Meet Stretch 3, an open-source robot that, according to its maker Hello Robot, heralds a future where versatile robots are in millions of homes. Originally introduced as a research platform, Stretch i [ ... ]



Sinusoidal Tetris
09/02/2024

This is just for fun and education, but it suggests that there might be other games waiting to be thought up along the same lines.


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Monday, 17 October 2022 )